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Association for Educational Communications 
and Technology 



WhatisAECT? 

AECT is the only national, professional association 
dedicated to the improvement of instruction through the 
effective use of media and technology. AECT assists 
its members in using technology in their jobs and to 
enhance the learning process. 

Who belongs to AECT? 

□ Media specialists 

□ Educators 

□ Librarians 

□ Instructional designers 

□ Corporate/military trainers 

□ Learning resource specialists 

□ Curriculum developers 

□ Television producers and directors 

□ Communications specialists 

□ Education administrators 

□ Others who require expertise in instructional 
technology 

What are AECT members Involved In? 

□ Hypermedia 

□ Interactive Video 

□ CD-ROM 

□ CDI 

□ Teleconferencing 

□ Film and Video Utilization 

□ Telecommunications 

□ Computer software and hardware 

□ Projection/presentation products 

□ Intelligent tutoring systems 

□ Videodiscs 

□ Distance learning 

□ And more! 

AECT’s publications, convention, trade show, and 
conferences present the leading edge on research and 
practical applications for these and other technologies. 

AECT History 

Unlike some other special-interest technology 
organizations, AECT has a long history with over 70 
years in educational technology. We’ve grown up with 
technology, advocating its integration into education 
from films to integration into education from films to 
overheads to interactive video and hypermedia. 



AECT began as the Department of Visual Instruction at 
the National Education Association in 1923, in the days 
when visual aids consisted of films and slides. In 1947, 
as educators were adapting technology used to train 
World War II service personnel for the classroom, the 
name of the organization became the Department of 
Visual Instruction (DAVI). Twelve years later, DAVI 
became an affiliate of the NEA and finally the 
autonomous association, AECT, in 1974. 

Today, AECT keeps an eye on the future of instructional 
technology while assisting educators with the changes 
and challenges that face them now. AECT members, 
now numbering 4,500, are professionals devoted to 
quality education. They care about doing their jobs 
better and want to embrace new methods, new 
equipment, and new techniques that assist learning. 

Membership in AECT increases your effectiveness, your 
expertise, and your skills. These qualities in turn 
enhance your professional image and earning potential. 

AECT Affiliates 

AECT has 47 state and 16 regional and national 
affiliates, and has recently established several chapters 
surrounding major universities and metropolitan areas. 
These affiliated organizations add a localized dimension 
to your AECT membership and allow for more 
interaction among your colleagues. For more details 
on chapters and affiliates in your area, contact the 
AECT National Office. 

AECT National Convention and InCITE 
Exposition 

Each year, AECT brings top speakers to exciting 
locations, and presents over 300 sessions and special 
events to provide the best training available in the use 
of media in education and instruction. The convention 
features the InCITE Exposition, the first trade show 
created exclusively for instructional technology 
products. At InCITE, you’ll see computers, learning 
systems, software, interactive multimedia, audiovisual 
products, films and videotapes, projectors and 
presentation products, video equipment, accessories, 
and more. The convention offers tracks of sessions 
focusing on specific interest areas surrounding AECTs 
nine divisions and other special interests. The 
convention has featured a Hypermedia Strand and a 
Total Quality Management Track. In addition, intensive 
full and half-day workshops are offered for in-depth 
training on the latest technology applications for 
education. 

Research and Theory Division (RTD) improves 
the design, execution, utilization, evaluation, and 
dissemination of educational technology research and 
theory; advises educators on using research results. 
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Jeroen Breman 
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Abstract 

Problem-centered project work, involving groups who work and learn collaboratively, is an important form 
of instructional organization. This is especially so for students learning about the design and development of 
multimedia learning materials, in that working as members of design teams will be an important part of the 
professional futures of many of the students. At the University of Twente in The Netherlands, students in the 
Faculty of Educational Science and Technology have an initial experience in the multimedia-design process as an 
integral part of a first-year required course. In this paper, we identify key issues in the instructional design of the 
course itself, illustrate how information-technology is not only studied in the course but also used as the learning 
environment, and in particular show how the WWW provides an integrated setting for the students, so that they can 
experience as learners the sorts of environments they are studying about and designing in their groups. An extensive 
evaluation of the first cycle of the course (September 1 995- June 1996) was carried out, and its results integrated into 
the 1996-97 cycle. In particular, stress is being given to the integration of process and product, to evaluation as a 
key unifying process throughout the course, on the use of “productive communication ”, our emphasis on learning 
how to learn, and on the implications of the new roles for both students and instructors. In each of these categories 
the course ISM-1 WWW site plays a critical role in the “pedagogical engineering ” of the course which is occurring. 
The course site can be accessed at http://www.to.utwente.nl/ism/isml-96/home.htm 

Group-Based Cooperative Learning for Multimedia Design Education 

Although the importance of instructional methods that foster learner self-reliance and that support 
cooperative learning have been long established, their combination in a course for problem-oriented multimedia 
design education is less familiar. Learner self-reliance has many aspects, including self-awareness, “actively willing 
to seek feedback from others and able to give constructive feedback”; skill in action planning and implementation, 
such as the ability to organize time effectively, and to monitor and evaluate progress against specific criteria; and the 
development of the habit of reflection, learning from one’s own experiences as well as those of others (Hawkins & 
Winter, 1996). Contributing effectively to group-based cooperative learning situations involving working together to 
design, develop, and evaluate a solution to a complex problem, requires all not only self-reliance skills on the part of 
each individual group member, but also success with additional aspects such as consensus about the group’s goals, 
individual responsibility toward the communal attainment of those goals, efficient procedures for group self- 
management, and group cohesiveness and internal positive spirit (Slavin, 1990). Such problem-centered, cooperative 
instructional orientations are seen as critical for the development of life-long learners (Casey & Tucker, 1994), and 
perhaps one domain where life-long learning will be most important is that of the educational technologist. The 
rapid emergence of new forms of media and new communication channels is leading to an explosion of new forms of 
media products, including environments and tools as well as structured learning resources for hyperlinked multimedia 
distributed networks (Collis, 1996). The educational technologist (or instructional technologist) will be increasingly 
challenged to provide professional leadership in rapidly changing technical and pedagogical environments, and thus 
life-long learning will be a job requirement (Gustafson, 1993). 

There are some examples in the literature of descriptions of courses for prospective educational 
technologists that are designed around this kind of future-looking, group-based, project orientation for interactive 
multimedia design and production. For example, Liu (1996) at the University of Texas at Austin uses a collaborative 
learning approach to “simulate real-world multimedia production” with her graduate students and notes that “the 
authentic aspect of the learning experience motivated many students because they perceived it as helpful to get them 
better prepared for the job market. The collaboration and interaction among groups enhanced students’ understanding 
of the multimedia technology” (p. 787). Liu and Rutledge (1996) also extend this approach with much younger 



learners, secondary students, also with positive results in terms of both process and product. But regardless of the 
maturity of the learners, the approach is challenging to implement in practice, particularly for the instructor, who 
among many responsibilities must monitor appropriate time management by the learners, find an efficient and 
effective method for feedback and motivation, and foster knowledge construction as well as product production (for 
example, see Liu & Rutledge, p. 400). 

General Description of the Course “ISM-1”, Cycle 1 (1995-1996), and Cycle 2 
(1996-1997) 

At the University of Twente, students in the Faculty of Educational Science and Technology work within a 
problem-solving framework throughout their study. In fact, the Twente approach to educational technology is that of 
“an engineering approach to educational problems’. Within the Faculty, one department specializes in the design, 
development, and evaluation of educational media products, what the Dutch call “instrumentatietechnologie”, or 
literally, the technology of learning instruments. This department is known internally as “ISM” and thus its required 
course for first-year students is called “ISM-1”. The course is regularly refreshed and revised, to reflect the rapidly 
changing nature of educational media. In the previous few years, considerable attention has been given to moving 
from older forms of media such as audio-slide presentations, text- and graphics- based computer simulations and 
tutorials, and interactive- video products to computer-based multimedia. In 1996, the increasing educational use of 
distributed networks to access multimedia resources, particularly via WWW protocols, means that instrumentation 
for these sorts of situations needs to become part of the students’ experience. In particular, we have three general 
goals for our students: that they learn about educational media, its implementation possibilities and its effects in its 
full range of forms; that they gradually become competent as professionals in the overall design and life-cycle 
process of learning materials; and that they base this learning, not only on theory but also on practice. Also, the 
staff has an additional goal: that we practice what we preach, and therefore that students gain critical insights into 
educational media through actually learning with such media themselves. 

The course “ISM-1” is the first initiation of the students to the field of educational media and to the 
particular practice and philosophy of the Department ISM. Following is a brief description of the course. In 
particular, we refer to the two most- recent cycles of the course, that of 1995-96 (referred to as Cycle 1) and the 
current 1996-97 (referred to as Cycle 2). 

General goals. 

There are four major, intertwined sets of goals for the course, relating to its: 

-Content and theoretical framework: To introduce students to major forms of educational 
instrumentation, to a model for the lifecycle of media products, and to terminology and concepts 
relevant to each of these phases as well as to different forms of educational instrumentation . 

-Practical experiences with software tools and media-related equipment: To introduce students to a 
broad range of professional tools and equipment for the design and development of educational 
instrumention, and to scaffold students in their use of these resources in their own project experiences. 

-Group-related goals: Critical to the course is the experience of working as a group, toward the 
solution of a realistic problem. The “Jigsaw Method” (Aronson, Blaney, Stephen, Sikes, & Snapp, 
1978) is used in the course. In the Jigsaw Method, every student has his or her own unique 
responsibility to the group; like a piece in a jigsaw puzzle, the result can not be whole without all 
these unique pieces fitting well together. 

-Individual goals related to the process of becoming a professional: The research from groupwork emphases not only 
the importance of group goals and group interrelationships in social constructivism, but also the importance of the 
individual in his or her own learning and development (Jonassen, 1995). In ISM-1, important goals relate to the 
individual learner him- or herself. Learning how to learn, learning to take self-responsibility for one’s own study 
habits, learning how to be an effective self-evaluator, learning how to ask questions and how to find help after one 
has used one’s own resources for answers; all are important. 
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Course organization 

The course is organized around three problem settings, each of which motivates a group project extending 
over a trimester (12 weeks). Each project in turn is based on a realistic problem. Students are organized in groups of 
eight or nine, where each group has a Manager, and each pair of students has a “specialism” which they must bring 
to the common solution of the problem that the group is confronting. Each group is responsible for contributing a 
component of an overall solution to the given problem. For example, the problem for the first project of ISM-1, 
1996-97 cycle, was to help first-year students in the faculty (in fact, themselves), to find information about different 
problems which they might encounter as students new to the university, problems with housing, finances, personal 
relations, health, study skills, choice of study, emotions and stress, and to have a convenient way to make contact 
with appropriate persons who can help them further. The WWW was chosen as an appropriate instrumentation form 
for such a communication and information system. Each group was responsible for designing and constructing a 
suite of WWW pages relating to one of the problem areas, and together all eight sets of WWW pages would 
comprise the overall solution. 

Time commitment 

Each group is expected to work together for four hours per week, during a scheduled time but generally 
without an instructor present. In addition, each student individually is responsible for staying on tempo with the 
study materials of the course, approximately two hours per week, and for attending all of the five scheduled lecture 
sessions per project. The total over the full year should relate to 240 hours of participation time. 

Determining the course mark. 

The final mark for the course consists of the average of the three group-project marks (with in general all 
members of the group receiving the same mark) and the scores on three written tests, one after each project. Students 
must have at least a passing mark in each of these for the overall mark to be averaged and awarded. 

Integrating the course within an electronic environment. 

In the first cycle of the course, the FirstClass computer conferencing environment was used not only for all 
course communications but also for course and project organisation and for lecture notes. There was a separate 
textbook as well as various sets of handouts for the students which they were expected to keep in a binder. In the 
second cycle, taking insights from an extensive evaluation carried out during the 1995-96 cycle (Breman & Carleer, 
1996) all course resources have been redesigned within a single WWW-based course environment. Instead of a 
textbook, all reading materials are included within this environment. This WWW environment is discussed 
extensively further in this paper. 

Evaluation study and results 

Rather than giving more overall description of the ISM-1 course, we will instead focus on six major 
attentions points emerging during the 1995-96 cycle and its evaluation, and describe the evolution of the 1996-97 
cycle of the course relative to these attention points. The evaluation was commissioned by the Faculty because of 
the innovative aspects of the course for first-year students. Could these young students work without direct 
supervision? Could they handle group work, combined with self-responsibility? If the emphasis was on process, 
rather than product, what would be the norms for satisfactory achievement? An external evaluator was appointed to 
work closely with the Course Team throughout the 1995-96 year, as an observer and also as someone offering on- 
going comments and impressions. The evaluator also observed and interviewed students as they worked on the 
projects, interacted on a personal basis with different groups of the students in the course throughout the year, and 
prepared a written analysis after each of the three projects as well as a final report (Breman & Carleer, 1996). 

The final conclusions of the evaluation study reflected what the student achievements had already shown 
over the full 1995-96 cycle: That the group approach worked well, that the tasks and assignments were successfully 
completed with much motivation and enthusiasm (every student but one completed the course successfully), and that 
all involved in the course worked together responsively and creatively to learn from each experience in the course and 
to adapt and evolve the course accordingly. Similar results have been found from the first project of the 1996-97 
cycle (See the appendix ). 

Given this overall, the remainder of the paper discusses major aspects of the course, and its WWW-based 
learning environment, in more detail. 





7 



Relating Theory To Practice: The Integrated Course Site 

In 1996-97, the entire course is integrated within one WWW site. This site can be visited at: 

http://www.to.utwente.nl/ism/isml-96/home.htm 

The site for Project 1 consisted of many hundreds of pages, developed by the Course Team (and by the students 
themselves, as will be described later). For a sense of consistency throughout the site and to help the students realize 
when they were still within the course site and when they were in an external link, a course logo reappears on most 
internal pages. Figure 1 shows the logo. 




Instrumentatie 

Technologie 

FACt^TY OF 8DUCATK5NAL SCIENCE AND TEC^OLOGY 
UNIVERSTYOFTWBtfTE 



Figure 1. Logo to identity internal pages in the ISM-J course site. 



The “Week-by-Week” and “Study” Centres. 

The integration of theory and practice is always a challenge, particularly in project-oriented courses. 
Activities cannot be metered out as practice exercises following a section of theory, as can be the case in a 
mathematics course, but must occur as the higher-level culmination of many different insights. Theory must be 
translated and applied to practice. The colleges and study materials in ISM-1 are meant to make this translation as 
explicit as possible. Thus, based on the experiences of the first cycle of ISM-1 as well as research occurring at TO 
and elsewhere, the decision was made to integrate all aspects of ISM-1, theory and practical, within one hyperlinked 
environment where explicit linkages can be made between weekly group work and weekly study materials. Figures 2 
and 3 show the “Week-by-Week Centre” and the “Study Centre” of the ISM-1 Course Site to illustrate how the 
materials are presented to the students. Within each link of these centres, explicit hyperlinking occurs between the 
concepts in the Study Materials and College notes, and the assignments for the group activities for the week. 
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Figure 2. The “Week-by-Week Centre”, linking to study materials and assignments 
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Figure 3 . The “Study Centre”, giving access to the theoretical materials of the course 



From bound to on-line textbook. 

In the first cycle of ISM-1, there were some complaints by the students about the textbook and its 
relevance, and about the lectures and their relevance. The Course Team was also not pleased with the fit of the 
textbook to the projects, even though it was the best textbook that could be found. The unique aspects of the ISM-1 
course mean that no single externally produced textbook is likely to be available, appropriate for the level of the 
Dutch first-year students and capable of being closely related to each week of their group activities. Thus the decision 
was made for the second cycle of ISM- 1 that the Course Team would write the theoretical materials for the course, in 
a form appropriate to both the group activities and also as a example to the students of how hyperlinked study 
materials (about which they will study in other courses in ISM and TO) can be used as learning resources. 

The Course Team sees these hyperlinked study materials as having the following benefits: 

• Capable of being fine-tuned to respond to each week’s group experiences; thus to serve as both theoretical 
material and feedback from the Course Team to the class as a whole 

• Capable of being linked to multimedia examples, both of student work and from outside WWW sites 

• Capable of including self-testing and evaluation questions via JavaScript and CG-I forms, to stimulate 
interactivity while self-studying and to collect student responses to evaluative questions 

• Capable of showing, as explicitly as possible, how the colleges, the study materials, the course feedback, and 
the group activities all are intended to be interrelated 
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Evaluation work in Scotland (Brown, Doughty, Draper, Henderson, & McAteer; 1996) indicates that the 
effective integration of resources into higher-education courses must be done in such a way that students are 
motivated to make use of the resources, that the resources should be easy to access, that students must know the 
resources are available to them, and that the instructors must communicate to the students why the resources are 
relevant. Students are practical, and “may ignore anything not felt to be essential... use is often related to the time 
available to use them and the problems associated with doing so”(p. 111). By integrating the study materials with 
the course communication, the marking in the course, and the project work, all within the same WWW-based 
environment, and emphasizing the value of these study materials to the weekly activities, we hope to increase the 
likelihood of careful use and study of embedded resources. 



Integration Of Process And Product 

Important to the course is the integration of support for the group processes of the course, not only with 
the study materials but also with the products that the students design and produce. The WWW environment allows 
this integration to take place, at least for some of the products that the students will produce. Figure 4 shows the 
“Group Centre” environment within the Course Site for Project 1 of 1996-97. From this page, students can find the 
composition of their groups and their specific tasks, the managers of the groups were given structured group-report 
forms to complete on various occasions, the self-evaluation of the group relating to their own products was always 
available for reference, and the products themselves could be seen and evaluated by the Course Team and by the 
students in all of the other groups, as regular evaluation activities throughout the project. We discuss this process 
more in the next section. 
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Figure 4. The “Group Centre ”, showing links to Manager inputs, self-evaluation comments by each group, and 
giving access to the current stage of each group’ s products. 



Evaluation As A Key Process 

Not only because of important of developing self-evaluation skills in students as part of their growth toward 
more self-reliant learning (Hawkins & Winter, 1994) but also because of the more-general importance of on-going 
evaluation in both the multimedia lifecycle and the lifecycle of a course, we are working hard to transform 
evaluation from something given out at the end of a process by the instructors to something ongoing throughout the 
project, and for which the students themselves have the major responsibility. 

Formative evaluation as a key to integrating theory and practice. 

To accomplish this more effectively in the 96-97 cycle of the course we are utilizing a number of methods 
(Collis & Meeuwsen, 1997). One of these which is emerging as particularly versatile is that of making use of the 
capabilities of the WWW environment to allow us to embed CG-I forms to structure group self-evaluation and peer 
evaluations, and to base these evaluations not only on the concepts about formative evaluation developed in the 
Study Materials, but also to make the criteria for the evaluation be the set of “design guidelines” for the course. 
These design guidelines are developed week by week in the Study Materials, illustrated with examples from both 
external links and during the lectures (called “colleges” in Dutch), and used repeatedly as the criteria for formative 
evaluations of the group products. The design guidelines are accumulated in the Resource Centre of the Course Site. 
Twenty of them were developed during Project 1. The Managers were directed during Week 44 to lead their groups in 
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self-evaluation of their sites, and to submit a report, via CG-I forms, in which they selected two of the guidelines 
that they felt were well met by their work and two of the guidelines for which improvement should be made. These 
self-evaluation reports became part of the Course Site (see Figure 4). 

Peer evaluation. Following the self-evaluation, came peer evaluation. Figure 5 shows an excerpt from the 
Assignment materials for Week 45 of the course in which the formula to be used for peer (and later, Course Team) 
evaluation was presented based on the 20 design guidelines and the specialist responsibilities. 
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During Week 45, you need to (1) finish your sites, and (b) carry out a formative evaluation of each 
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You will still have time for last improvements in Week 46, making use of the formative evaluation 
results which you will receive from the other groups during this week (W eek 45). 
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Figure 5. Relating the design guidelines to specialist responsibilities and to the peer evaluation 



Figure 6 shows the peer evaluation form, available via the WWW site as part of the Assignment for Week 45 of the 
course. The instructions for the week were that the Manager of each group was to organize his or her group so that 
members of the group evaluated each of the other seven group sites, and filled in the formative evaluation of that 
site. These forms were submitted (by a click on the submit button), sent to the student assistant who in turn 
forwarded them to the Manager of the group being evaluated and to the Course Team. It was in turn the Manager’s 
task to discuss the peer evaluation comments with his or her group and to lead the decision making about how to 
revise their sites based on these evaluative comments. 
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Figure 6. CG-I form for peer evaluation (and later for Course Team evaluation) 

Evaluation as study material for the examination 

But the integrative role of formative evaluation throughout the course does not stop with the peer 
evaluation. The Course Team told the students very clearly that the final evaluation of their sites would be based on 
not only the Course Team’s use of the same evaluation forms, but also that the peer evaluation would be taken into 
account. Finally, the examination to follow the first project contained a major section relating to the evaluation 
process and its results: asking the students to reflect on points of agreement and disagreement between their own self- 
evaluations of their work, the evaluations of their peers, and the Course Team evaluations. Students were warned to 
expect this question; all feedback from the Course Team was placed in the Course Site (in the Group Centre) the day 
after the final presentations of the projects, so that the integration of theory and practice, through the on-going 
process of evaluation based on design guidelines, could be reflected in the written test for the project. 

Through these processes, we believe we are making significant progress toward not only integrating the 
theory and practice of the course but also toward helping students develop insight and self-responsibility for their 
own evaluation. Evaluation should be seen as part of the on-going process not only of the course but working as a 
design professional. (A walk-through of all of the above evaluation-related aspects of the course can be followed by 
the WWW-site “guided tour” available at Collis & Meeuwsen, 1997). 
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Communication: Multiple Forms And Purposes 

Students can see and hear from all course team members at the lecture sessions and can of course present 
questions but in a room of 85 students question-asking is only something that a few students per session can much 
contribute to. The Study Materials can be seen as a different form of one-to-many presentation, containing not only 
conceptual matter but occasionally personal reflection from the Course Team. Less-formal messages to the whole 
class are also important. The major example of these is the “First Stop” page of the Course Site, indicated by an 
ever-present icon (a lightening bolt) in the top left-hand corner of the navigation frame. The students know that it is 
their responsibility to check First Stop first thing on Monday morning of each week. There they find an overview of 
the week: what is new in the Course Site, what they should read and do for the week. This message is kept brief; its 
intention is to point the way to other messages that give detailed communication about weekly activities. 

“Productive communication” 

An major innovation in ISM-1 1996-97 compared to the 1995-96 cycle is the use of CG-I forms for 
“productive communication.” With a CG-I form, input expected from the student can be highly structured (see the 
peer evaluation form in Figure 6) or can still be as open as an e-mail message. This allows us to indicate clearly to 
the students what input is expected, and to make it as easy as possible for them to enter this input in a consistent 
fashion. The student simply types directly into the Course Site. During the 1995-96 cycle of ISM- 1, the inability to 
create structured input forms in the computer conferencing environment led to two less-than-ideal situations: Either 
students replied via an unstructured message, and thus were not always sure of how much was expected of them in 
responding, or the Course Team had to switch to separate paper forms when structured input was critical to providing 
guidance. Also, the flood of communication coming from the students, with many different reports and summaries, 
led to an unmanageable amount of mail for the Course Team to effectively read, handle, forward, and archive (there 
were over 3,000 e-mail messages of various sorts after the total 1995-96 cycle). 

But it is not the ease and clarity of expectation that makes CG-I-form communication most important in 
ISM-1. It is the fact that students’ communication can be clearly seen to serve a purpose in the Course Site, in fact, 
to become part of the study material for the course. (We have already mentioned how this is critical to the on-going 
evaluation processes in the course). This “productive communication” occurs by entering the content of the students’ 
CG-I-input into HTML form, linking it to the Course Site, and then making use of it in the instructional activities 
of the following week or weeks. In this way, students can see that entering their input, on time, has a function to 
the group as a whole, because it becomes part of the learning material for the group. It also helps students to develop 
their self-evaluation skills, in that seeing what other students have entered and comparing it to one’s own entry can 
be a stimulus to self-monitoring. 

An example of the use of productive communication within ISM-1 can be seen from the Specialist Centre 
page (Figure 7). After the Specialist Sessions for each of the roles in Project 1, students were required to input, via a 
CG-I form, a brief summary of the main points to remember from the session. These summaries were merged into 
one HTML page, and then were always available from the Specialist Centre page. When students later had a question 
relating to any of the specialist roles, they could turn to the “Specialists’ Summaries” as well as to the “Bookmarks” 
provided to them by the Course Team, as their first line of help. Figure 8 shows a portion of the input from the 
HTML specialists, available from the Specialist Centre page. Da Bormida, Donzellini, and Ponta (1996) also have 
commented on the importance of students becoming involved in the creation of their own learning material, “under 
properly defined rules, students are at the same time, users, producers, and evaluators. ..”(p. 160). The WWW 
environment in ISM-1 makes this productive communication convenient, as well as accessible to all. 
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Figure 7 . The Specialist Centre , Wi 7 /t learning resources for the various roles, and also with links to CG-I forms for 
Specialist reports. 
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Summary group 4 by Annechien Langevoord & Marianne 
Polderdijk 

INHOUD: 

1 . Het principe van HTML 

2. Enkele verwijangen naar Web-pagina's 

3. lets over BBedit 

ad 1. HTML is de afkorting van HyperText Markup Language. Hiermee kunje door het gebruiken 
vanvastgelegde codes eenpagjna-opzetmaken. Door onder 'View' de optie 'document source' te 
kiezen kunje de pagjna-opzet van elke pagina bekijken. Hier volgen enkele voorbeelden van codes: 
TITLE titel invullen /TITLE, B tekst die vet gedrukt moet worden lb. Zie voor nog meer 
HTML-codes verwijzing a. bij 2. 

ad 2. Enkele verwijangen naar Web-pagina's: 

a. Bare Bones Guide to HTML 

b. HTML: Hypertext Markup Language 

c. InfoServices HTML info 

d. Infinet Colors 

ad 3. BBedit is het programma waar de HTML-codes in verwerkt zijn zodat je eenvoudig een opzet 
kunt maken. Je komt in BBedit door onder het Apple-menu Editors te kiezen en daarin BBedit lite 
3.5. 1 aan te klikken. Voor een stan- daard codering voor het beginnen van een pagina kies je onder 
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Figure 8. Summary of the specialist reports , available as a study aid for HTML coding. 

Toward Self-Responsibility: Learning How To Learn 

An important part of the philosophy of ISM-1 is scaffolding the students toward self-responsibility . The 
example shown in Figure 8 is typical of this philosophy. If students have a question, they must learn a series of 
strategies to find help themselves. The Course Team has the responsibility to make as much help available as is 
useful, such as seen in the Bookmarks and Specialist Summaries in the Specialist Centre page (Figures 7 and 8). But 
the students must learn to look for help in the Specialist Centre, and in other places in the Course Site. After this, 
the students must learn to ask each other for help; this is part of the peer teaching philosophy of the course and 
consistent with Vigotsky’s theories of peer support. Students also have available to them examples of other 
students’ work (through the Group Centre page, for example), and should learn to look at those examples for help. 
After this, the Communication Centre is always available, and a message sent to the Course Team members will be 
answered within the day. But of course, unaccountable problems do arise, often related to the network or something 
like a room being unexpectedly locked. Students must learn how to classify their problems, and how to identify 
which require immediate human intervention (unlocking a door) compared with those that require patience and regular 
checking (waiting when the network is down). 

Despite the efforts that were made in the 1995-96 cycle of ISM- 1, the comment was expressed in the cycle 
1 evaluation report that students needed (or wanted) a clearer sense of where to get help. Thus the 1996-97 Course 
Site is being carefully designed to make adequate help clearly available. However, providing adequate on-line help, 
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and supporting students in their growth toward being self-reliant and good peer supports for each other, is not a 
simple task, and will continue to be an action area for the Course Team. One idea we hope to develop is a question- 
and-answer component in the Course Site, where we can build on students’ questions and evolve these into an 
“answer web” (Smeaton & Neilson, 1994) that students can consult, and to which they can contribute. Another focus 
is to continue to try to help the Managers develop more-effective managerial skills, not an easy task for students just 
out of secondary school (or for professionals, as well; see McDaniel and Liu, 1996, for a good analysis of the skills 
and insights needed for good management of multimedia development projects). In the “Group Planning Pages” for 
the second project of the 1996-97 cycle of ISM-1, we are now using a WWW-based approach to a project- 
management tool to help Managers more in their tasks. 

The Evolution Continues 

The course ISM-1 is a learning experience for all involved, and thus we can identify many areas in which 
we will continue to pay particular attention even after the 1996-97 cycle. A research project is already underway 
focusing on helping students become better at managing their own work and helping the instructors to have adequate 
information about the students’ work so that appropriate feedback can be given “just in time” (Van der Veen, 1996). 
Another research project will begin in 1997 in which scaffolding will receive major attention: how can scaffolding 
be effectively incorporated, and withdrawn, for students working more and more independently with WWW-based 
environments, such as the group project work in the course ISM-1? (Collis & Verhagen, 1996). Another important 
focus is peer teaching: How can we help the students to help each other in increasingly effective ways? (The 
“intelligent coach” to support the Jigsaw Method in teams of computer science students working together on 
complex programming tasks which has been developed by McManus and Aiken, 1995, is an example we wish to 
explore). The long-standing issue of media selection is also important to us: The integrated WWW site offers many 
benefits compared to non-electronically linked course materials, but the important benefits of other media, such as 
printed text, need to be also considered. Perhaps not everything should be in the WWW site, even if it is digitally 
possible, but by what criteria do we decide? 

A particular focus is that of integrating multimedia resources in the course site. As one aspect, how can we, 
as Da Bormida and his colleagues suggest (1996), make increasingly effective use of the students’ own work as 
examples within the Course Site? Partly this is a technical challenge. We are already exploring the integration of 
CD-ROM materials with the Course WWW site during the 1996-97 cycle in order to better utilize the video products 
made by the Cycle 1 cohort. In addition, in order to continue our emphasis on linking theory and practice, we would 
like to be able to link videotaped fragments of the lectures to appropriate points in the students’ group work and self- 
study. To do this, we are planning for a video-on-demand component to eventually become part of the course site. 
Integration of multimedia resources, either from the students’ work or from videotaped fragments or the lecture 
sessions, is by no means only a technical issue. The instructional issues and design issues involved in selecting 
appropriate video segments of appropriate lengths, indexing them effectively, and pointing out to the students how 
to apply those examples to their own work will occupy our attention, as designers, instructors, and researchers, for 
many years to come (Verhagen & Blanke, 1996). 
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Appendix 

Evaluation feedback from students, Project 1 of 1996-97 cycle of the course “ISM-1” 

Note: The evaluation results were submitted by the students in the 1996-97 cycle of the course (n =85) via a CG-I 
script within the course WWW site early in the first project and after the completion on the first project, as a paper- 
based questionnaire completed during the first examination. For each item, there was a five-point scale, with “1” 
labelled as “strongly disagree”, “2” as “disagree somewhat”, “3” as neutral, “4” as agree, and “5” as strongly disagree. 
The following compares some of the responses at these two time periods. 



Questions (n = 80 students responding, out of 88) 


Near the start of 
Project 1 (Sept. 
96), median score 


After completion of 
Project 1 (Dec 96), 
median score 


1. The communication between the group members is 
sufficient. 


4 


4 


2. My group has maintained a good division of tasks. 


4 


4 


3. The efforts of each individual group member on the 
project work are comparable. 


3 


3 


4. I was able to work by myself on my tasks without 
help. 


4 


4 


5. The support from the staff members is sufficient. 


4 


4 


6. I enjoy working on ISM-1. 


4 


5 


7. 1 like the specialist tasks that I have in my group. 


4 


4 


8. I like the topic our group is working on. 


4 


4 


9. I am satisfied with the learning effect of my work so 
far. 


4 


4 
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Abstract 

High ability , highly verbal fifth grade students were the selected subjects for this study. The purpose of the 
study was to identify students' use of learning strategies, encoding process , and navigational decisions in a 
hypermedia lesson on propaganda techniques by employing think aloud protocols. Subjects were asked to read and 
think aloud as they worked their way through the concept lesson and immediate posttest. These think alouds were 
transcribed and coded for analysis. Results revealed a wide variation in amount and type of learning strategies used by 
students with high, average, and low test scores. Navigational decisions and encoding processes also varied widely 
among these groups of students. Noted that high test scores tended to use more and more varied learning strategies 
as well as being more consistent in their navigation decisions than the other two groups. The low test scoring group 
tended to use the encoding process of construction than did the other two groups; but with more errors or faulty 
construction. 

Introduction 

In recent years there have been amazing technological advances in the means for delivering instruction, 
such as multimedia and hypermedia programs. These advances allow educators to create more learner control over the 
instruction and has been widely touted that this flexibility is desirable (Duffy & Jonassen, 1992). However, such 
mediums lack structure and/or have ill-structured domains (Spiro, Feltovich, Jacobson, & Coulson, 1992). Thus, 
learners can become lost in these environments (Davidson-Shivers, Rasmussen, & Bratton- Jeffery, in press; Lin & 
Davidson, in press; Kerr, 1987; Shin, 1992). 

Students' success or failure in learning performance in such environments may be dependent on their ability 
to activate relevant strategies and monitor their own learning. Learning strategies are those mental techniques which 
provide for organizing, rehearsing and elaborating on knowledge as well as tactics for coping with motivation and 
affect (Mayer & Weinstein, 1986). The techniques can be expanded if metacognitive strategies, such as cognitive 
monitoring, are incorporated. Metacognitive strategies allow students to maintain self- awareness and control of 
their information processing when learning. 

There also seems to be a pervasive assumption among developers and educators that students often know 
how to manage their own learning in these environments (Davidson-, et al, 1996; Glaser, 1984; Federico, 1984). It 
has been found that effective learners can spontaneously generate and use specific strategies when interacting with 
traditional instructional materials (Anderson, 1980; Rohwer, 1980). However, Battig (1979), among others (Jones, 
1987; Davidson, 1988), states that there is wide variation in learners' abilities to generate and use learning strategies. 
In reality, not all learners are sophisticated in their learning strategy generation and use. In fact, research has shown 
(although often ignored) that students make poor choices in determining what is best for them within a given 
instructional environment (Clark, 1984; Cronbach & Snow, 1977). This may be especially true within hypermedia 
in which navigation adds an additional level of complexity. Davidson and associates (1995) found that middle school 
students who used complex learning strategies of cognitive monitoring, elaboration in combination with rehearsal 
strategies did better than those students who indicated that they were only using rehearsal strategies. Successful 
performance in the hypermedia lesson was also based on more strategies being used and in greater variety by these 
students. 

Related to these learning strategies are the navigational decisions for determining what part of the program 
to see, how much of the program to see, and when. Davidson and associates reported in that same study that 
students tended to seek additional items based on how well they believed they knew the new information. In most 
cases, these middle school age children tended to move through the program after two or less practices because they 
"thought they knew it." However, the trekcmefit materials were a hierarchical-based structure in which students 
tended to select materials in the order by which they were presented. Based on the results of this preliminary study, 
it was'posited that effective users of the hypermedia learning material employed appropriate strategies and decision 
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making techniques. However, it is not necessarily clear what strategies would be employed by such students if the 
hypermedia lesson had another type of linking structure that was web-based. 

Purpose of the Study 

The primary purpose of this current investigation was to determine the various strategies that students 
employed in a hypermedia environment that had an associative structure (Lin & Davidson, in press; Jonassen, 1988). 
In order to discover the strategies that students employed, we decided once again to use a think aloud protocol method 
with high ability and high verbal students. 

The purposes of this study were to identify: 

Those learning strategies generated by the individual student in an associative-linked form of hypermedia 
environment. 

The commonalties of heuristics generated by the students who performed successfully on the posttest. 

The navigational decisions and points of departure (if any) that these students employed when moving 
through the various parts of the program. 

Subjects 

Subjects were 5th grade students (N = 12) from both public and private middle schools in the Mobile, 
Alabama area. This was a non-randomized pool of subjects due to the fact that the researchers asked teachers to select 
high ability, high verbal students to participate. It seemed reasonable to assume that these students would have 
acquired effective strategies for learning new information as well as communicate their thoughts than other students. 
All participation was voluntary and confidential. 

Materials 

The lesson, Propaganda Techniques, is a computer-based coordinate concept lesson which describes various 
categories of propaganda techniques used in advertising. The advertising screens were modeled after advertisements 
popular magazine and television ads. The original lesson was created by Carrier, Davidson, & Williams (1984). The 
current version was redesigned using HyperCard 2.1 software for use with the Macintosh LCs or PowerBooks. This 
newest version allowed students to select any of the four concepts and once within the program, they could select any 
example item, practice item, or definition. In addition, they could jump back to the submenu or the main menu on 
any given screen. Although not totally random, this associative linked structure allowed flexibility in amount seen 
and the order in which items were seen. 

Procedures 

Researchers were trained to monitor observations, to conduct interviews, and to encourage think alouds. 
Training occurred through written descriptions and articles on these qualitative methods, followed by discussion over 
the readings, and a demonstration of these techniques was modeled during a pilot test of the HyperCard stack. 

Teachers at two schools were asked to suggest potential subjects who possessed high ability and high 
verbal. We sent out permission slips to all students matching this criteria and their parents. Written permission to 
participate in the study was obtained from seventeen students and their parents. We allotted two hours for each 
subject to complete the lesson, the test, and the interview. However, most subjects completed the lesson within an 
hour and a half. Subjects were scheduled on an individual basis to proceed through the lesson. 

For each subject, experimenters explained the general purpose and procedure of the study. Subjects were 
told that information would be kept confidential, that they were to talk out loud as they went through the lesson. In 
turn, they would be videotaped as they proceeded through the lesson and that the tapes would be destroyed at a later 
date. The experimenters described the lesson and stated that the students would be tested over their knowledge of the 
concepts presented. 

As each subject worked through the lesson, the qualitative procedure of "think alouds" (Flower & Hayes, 
1981; Wedman & Smith, 1987, Davidson, et al, 1995) was used in order that subjects explain what they were 
thinking, how they were remembering, and why they were making the selections from one section of the program to 
another. If the individual fell silent for any length of time during the session, the experimenter would prompt them 
by asking questions such as "what are you thinking?", "why did you choose that?", or "how did you decide that?" As 
each student completed the program, his or her responses were video- and audio-taped. Experimenters observed 
students as they progressed through the lesson and made additional notations of navigation and/or use of strategies 
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such as counting on fingers, moving lips, and other body language cues. Each subject completed a sixteen-item test 
at the end of the lesson and were interviewed, which was also videotaped. 

Data Analysis 

Data considering generated strategies and their use were collected through the students' verbal reports made 
before, during, or after performing the instructional task (Gagne, 1985). The reports were collected through the use 
of video/audio tape recordings of each individual subject (n=12). Upon completion of the task the subjects were 
interviewed for retrospective reports about specific subjects' actions during the instruction. 

The verbal reports were transcribed into written form (protocols) and examined for patterns of responses 
using a protocol analysis (Ericcson & Simon, 1984). Recorded data will be coded using a framework similar to that 
proposed by Weinstein and Mayer (1985). Each transcript was coded by two independent researchers, to ensure 
accurate coding of data. When discrepancies were encountered, they were resolved by the two coders. If they could not 
agree, a third researcher arbitrates and determines the appropriate code. Due to technical difficulties some video and 
audio tapes were unclear and could not be transcribed which reduced the number of students to twelve. 

The framework suggested by Weinstein and Mayer (1986) included eight categories of learning strategies: 
basic rehearsal strategies, complex rehearsal strategies, basic elaboration strategies, complex elaboration strategies, 
basic organizational strategies, complex organizational strategies, comprehension monitoring strategies, and affective 
and motivational strategies. (See Table 1.) Processes for encoding were also coded: selection, acquisition, 
construction, and integration. To these codes, the following were added by the researchers (Davidson et al; 1995; 
1996): experimentation effects (prompting, encouragement, design evaluation, directions, teaching, and clarification), 
student responses (clarification and evaluating design), and navigation decisions (confidence, curiosity, familiarity, 
avoidance, fun, order, practice, satisfaction, and unclear). 

Table 1. Categories of Learning Strategies, Encoding Processes , and Experimentation Effects 

Category Definition 



Learning Strategies 

Basic Rehearsal Strategies 
Complex Rehearsal Strategies 
Basic Elaboration Strategies 
Complex Elaboration Strategies 
Basic Organizational Strategies 
Complex Organizational Strategies 
Comprehension Monitoring Strategies 
Affective and Motivational Strategies 



repeating item names for remembering 
copying, underlining, shadowing material 
forming mental images or sentences relating information 
paraphrasing, summarizing, or describing knowledge 
grouping or ordering information 

outlining, hierarchy development 

checking for comprehension failures or for understanding 
student behavior and attitudes: alert, relaxed, interested, positive 



Encoding Processes 
Selection 
Selection Reading 
Acquisition 

Construction 

Integration 



attending to particular concepts presented 
attending to written information by reading aloud 
transferring information from working 
memory to long-term memory 
actively building connections between 
information reaching long-term memory 
actively searching for prior knowledge in 
long-term memory, transferring to working 
memory, and building external connections 
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Navigation Decisions 

Confidence 

Curiosity 

Familiarity 

Avoidance 

Fun 

Order 

Practice 

Satisfaction 

Unclear 



Experimentation Effects 



Student Effects 



expressed certainty of competence 
sought further information; aroused interest 
showed recognition of item or idea 

steered away from area/item due to some negative association 

expressed desire to be entertained; have fun 

decided based on some organizational pattern or to create an 

organizational pattern 

decided to try another item 

stated positive feeling about accomplishments 

reason for choice could not be determined; either student could not 

provide or could not be interpreted 

influence of experimentation in amount of prompting, 
encouragement, evaluating design, providing directions, 
and teaching 

asking for clarification and evaluating 
design of program 



Adapted from Davidson et al (1995; 1996); Weinstein & Mayer (1986). 

Upon completion of the coding process, the data was converted to text files and loaded into the software 
package HyperResearch (for either MAC version 1.56) which performs analysis on qualitative data. HyperResearch is 
designed to permit organization, storage, retrieval, and analysis of qualitative data (ResearchWare, 1991-1993). 
Studies using HyperResearch are structured by cases (individual units of study, in this research study, a case is one 
student) which are then comprised of source material. 



Results and Conclusions 

The preliminary study by Davidson and her associates (1995) identified a variety of patterns in strategies 
employed by individual student cases. For example, one participant always did two examples and then the test item; 
a second participant would work through any exercise in which she had right answers, once she made a wrong answer 
she went immediately to the test. Results also indicated that verbal learners were able to articulate their reasons for 
navigation selection and indicated that they preferred to proceed through a program linearly, if they knew in advance 
they were going to be tested. Reading and rehearsal were two of the most common strategies identified by 
observers/monitors as well as noted by students during interview. Based on interview data, it was found that a 
reliance on visual clues was also noted by many of the participants. These identified patterns will be further 
delineated at the presentations. 

The following graphs show the frequencies and types of learning strategies, navigational decisions, and 
encoding processes for high (score of 13 and above), average (score of 1 1 or 12) and low (score of 10 and under) test 
scoring groups. (Recall that all students were selected on the basis of high ability, high verbal.) Figure 1 shows the 
averages and types of learning strategies for these three groups. This finding shows that even among high ability 
students there is a wide variation in use of learning strategies. The average group (n=3) tended to use a combination 
of comprehension monitoring, elaboration, rehearsal and affective strategies more than the other two groups. For 
the high group (n=4), their combination of strategies were comprehension monitoring, elaboration, and rehearsal. 
The low group had some of the same strategies, but used them to a lesser degree and with more errors. That is, 
their elaborations or monitoring of their understanding tended to be miscued and inaccurate. Of interest, is that the 
average group used more strategies in greater combination than did the other two groups. Perhaps the high group 
may have been more interested in efficiency of learning rather than exploration since they knew that they would be 
tested on the concepts. 
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Frequencies and Types of Learning Strategies for High, Average, 
and Low Groups 
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Figure 1 



Figure 2 shows the frequencies and types of encoding processes employed by the same three groups. By far 
the greatest type of processing used was construction followed by selection. These experimenters suggest that 
construction refers to the process of someone taking information from working memory and transferring it to long 
term memory. Selection refers to concepts to which students paid particular attention while attending to the 
computer screen. While the low group tended to employ construction at a higher level than the other two groups, the 
experimenters noted during the lesson and in the protocols that the low group employed faulty construction by 
building concepts with flawed elements. The high group mainly processed the information at the higher levels of 
encoding processes, that of construction and integration. In fact, only the high posttest scoring group employed the 
integration level of encoding processes. 
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Frequencies & Types of Encoding Processes for High, Average and Low Groups 




Figure 2 



Figure 3 shows the frequencies and types of navigational decisions by the three groups. Again it can be 
noted that there is a wide variation in navigational decision patterns even among these high ability, highly verbal 
students, which may lead to their variation in posttest scores. The high group made their decisions based on 
confidence, order and practice. They were lowest in curiosity, familiarity, and fun. The decisions made by this group 
may suggest that these particular students were bent toward learning the concepts efficiently. The average group had 
similar navigation patterns in terms of confidence and practice but to a lesser degree than the high group. 
Conversely, the decisions made by this average group that were based on curiosity, familiarity, and fun are higher 
than the high group. The low group was lowest in confidence and practice, but higher in satisfaction than the other 
two groups. They also were highest in curiosity, familiarity, and fun. in comparison to the other two groups. The 
low group also made decisions based on order, but to a lesser degree than did the high group; however, experimenters 
observed that these decisions were less logical and were nonsensical than those in the other groups who navigated on 
the basis of order. Only the average and low groups made decisions based on avoidance. Overall, it appears that the 
more successful posttest performances were made when the learner based navigation on confidence, order and practice 
and less so if decisions were based on curiosity, familiarity, and fun. 




Frequencies and Types of Navigational Decisions Used by High, Average, 

and Low Groups 




Types of Navigational Decision 

Figure 3 



Based on observations in this current study, students tended to show a variety odf ways in what was selected 
and when. Rarely did students select the definitions, most selected a few examples and then moved to the practice 
items before preceding to the posttest. However, the choices varied as to which item was selected. In one case, a 
student selected only the practice items before going to the test. This particular subject never verified if he was 
correctly identifying the concept, instead he relied on the knowledge of results in making his choice. He was one of 
the lowest scoring on the posttest. In terms of navigation decisions, students successful in learning the concepts 
presented appeared to base their decisions on their understanding of the information as well as their level of 
confidence. These two navigational decisions mirror their use of those learning strategies. of comprehension 
monitoring and elaborations. 

Post interviews that immediately followed the posttest and knowledge of score yielded some interesting 
information. Most of the subjects were familiar with computers through school and/or having them at home. 
However, no one stated that they had encountered such a game. When asked how well they like the lesson, the vast 
majority stated that they liked it; some wanted to purchase the program because they thought it was fun and they 
wanted to improve their scores. They also thougth the lesson and test were about the right length. Although not all 
of the students mastered the concepts presented, they did appear to enjoy it. 



Significance 

This study was specifically designed to examine the variations in strategies that students employ. Because 
qualitative methods were used in this study, further analysis of the protocols will be necessary to yield experment 
effects on learning performance as well as strategy use and navigational decisions. Future investigations will 
examine strategy use within additional types of hypermedia and structures. In addition , other student populations 
will need to be sampled in future investigations. Also the identificatiaon of students’ thought processes with other 
types of learning ourtcomes may jpro vide additional information to developers and users of multimedia/hypermedia.. 
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Finally, it will be most important to determine how learners can be taught those learning strategies 
identified as effective for use in hypermedia environments. Learner control may become a viable option when we 
begin to understand which learner should have control, how much and what type of control should be given, and 
under what conditions should learner control exist. By determining effective strategies used and appropriate 
navigational decisions made in these various hypermedia environments, we can begin to understand how students be 
successful in learning performance. The significance of this work ultimately rests in further identification of 
effective strategies that have been generated and used and ultimately, in teaching these strategies to other students. 
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This paper provides a brief overview of Problem-Based Learning (PBL) as an instructional methodology. A 
master’s program in information and learning technologies , designed for classroom teachers using PBL as its primary 
instructional strategy is then described. Information about program issues , problems , and successes is provided. 

Problem-Based Learning 

Problem-based learning (PBL) is a student-centered instructional methodology that teaches content and skills 
within a knowledge domain by using substantive, carefully-crafted problems or challenges as the stimulus and focus 
for student activity (Boud & Feletti, 1991). Employing the principles from a constructivist view of learning, PBL 
prepares students to meet the challenges of their professions (Barrows, 1985, 1986; Bridges, 1992; Gijselaers, 1995a, 
1995b; Savery & Duffy, 1995): 

• Students acquire an essential body of retrievable and usable knowledge and skills; 

• Students develop problem-solving and reasoning skills that transfer to other situations; 

• Students develop lifelong learning skills (i.e., metacognitive and self-directed learning) to extend and 
improve their knowledge base in order to remain contemporary. 

To meet these educational outcomes, PBL uses problems, or challenges, as starting points for learning 
(Boud, 1985, p. 13). This is different from conventional classrooms which use problems as culminating activities 
after the teacher presents content. In a PBL environment, the teacher presents students with authentic, ill-structured 
problems before they receive any instruction. During the process of working on the problem, students build 
substantial knowledge bases through increasingly self-directed study. Students identify areas of learning to guide their 
own individualized study. The skills and knowledge acquired through individualized study are applied to the problem 
to evaluate the effectiveness of learning and to reinforce learning. Through collaboration with classmates, students 
refine and enhance what they know. When a solution is at hand, they present, justify, and debate solutions, looking 
for the best possible resolution to the problem. 

Content learning and skills development occur as natural consequences of solving the problem, much the 
same way people in the workplace learn on-the-job. In fact, in a PBL environment, students take on the roles of 
scientists, doctors, historians, programmers, engineers, advertising executives, mechanics, or any others who have a 
stake in the problem. Therefore, the students work with problems in a manner that fosters reasoning and knowledge 
application. 

University of Colorado at Denver Master’s Program 

The Information and Learning Technologies program in the Division of Technology and Special Services at 
the University of Colorado at Denver (UCD) offers a master’s degree program for practicing teachers. The primary 
emphasis of this degree program is to help teachers learn to integrate information and learning technologies in 
student-centered ways in their classrooms. The general features of the program include: 

• An extensive program design process including several iterations based on feedback from an external 
advisory group, 

• Instructional strategies focused on problem- and project-based tactics, 

• Classes offered on intensive weekends in six semester-hour blocks. Students meet for 15 hours on alternate 
weekends including Friday evening, Saturday, and Sunday morning, 

• Extensive use of e-mail during the two weeks between classes to discuss general issues and to communicate 
with subgroups, and 

• Most classes are held off-campus in technology labs in schools in which the students teach, enabling 
students to visit many different facilities. 

The content our on-campus master’s program was reorganized into six semester-long blocks of three to six 
credit hours. We took into account the feedback we have received over the past few years from classroom teachers in 
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our on-campus program, the advisory board, and our research on the skills, knowledge, and strategies that classroom 
teachers ought to have to be effective and integrated users of technology. The blocks are described below: 

• Technology as a Learning Tool (6 hours): Program orientation, constructivism, PBL philosophy and 

values, action learning, networking skills, HyperStudio, video production, the Internet and other electronic networks 

• Technology Tools and Issues (3 hours): Design and operation of networks, troubleshooting and 

management of technology, security and other tools 

• Integrating Technology in the Classroom (6 hours): Instructional design processes, rich environments for 
active learning (REALs), state model content standards, special needs, and integration of technology into the 
curriculum 

• Creating Learning Communities (6 hours): Knowledge-building communities and rich environments for 
action learning, WWW use for instruction, and web page design 

• Developing Interactive Media (6 hours): Principles for designing, creating and evaluating interactive and 
multimedia instructional hypermedia and multimedia materials for CD-ROM 

• Leadership and Change (3 hours): Diffusion and adoption strategies, fund raising, leadership skills 
Students are required to complete six additional hours through transfer credits, conference attendance, internships, and 
on-campus courses. Comprehensive exams are in the form of an individual master’s project and a master’s portfolio. 
Class time is spent on these projects during the final semester. 

From the beginning of development, the faculty in the ILT program made a commitment to using 
constructivist strategies in teaching the courses in the program. Our primary choice was problem-based learning. 
This brought about some unique issues as we learned to apply the PBL approach to a large group of people who 
only meet once every two weeks. We discuss those issues below. 

PBL and Large Groups 

Most PBL programs are conducted with small groups who meet frequently with tutors. We didn’t have that 
luxury. There are 20 people in the program and one teacher/tutor. The tutor is critical in helping students reflect on 
their learning and to remain faithful to the PBL process, so we had to learn how to conduct that reflection and 
monitor the process either in our absence or with a single large group. Strategies that we tried include: 

• Rather than meet with each group individually, we had the small teams prepare for reflection discussions by 

writing answers to reflection-based questions individually. Then we gave the teams time to discuss their 

individual responses together. Finally, we brought all the teams together to share their thoughts. 

• Infrequently (when there were two of us on site) we met with individual teams, mostly in an effort to 
monitor the process. Teams tend to fall victim to “production bias.” That is, teams tend to want to solve the 
problem, produce a product, or “win the race” and avoid things that get in the way of “producing” - things like 
taking time to reflect on their learning. 

• Since the students met only once every two weeks, we gave up significant portions of class period to team 
meetings and team work. To compensate for missed class time, we used e-mail discussions focused on reflection 
questions during the times between classes. 

• We gave reading assignments during class so that part of the class would have something meaningful to do 
while a team or individuals worked with a teacher. 

Even though we strongly supported the PBL approach and were prepared to make changes, giving up class 
time for other activities was (is) one of the hardest things to learn to do. Learning comes from working on the 
problem and from reflecting. Even after three semesters, we are still learning and thinking about how to handle class 
time. We’d feel more comfortable if we had more time to meet with the individual teams. 

Infrequent Class Meetings 

Most PBL programs have several meetings per week. We met once every two weeks. To facilitate the PBL 
approach, we gave up significant amounts of class time for small group meetings. This continues to pose problems 
for us. The teams need to meet together more frequently and the distances several of the students had to travel made 
this difficult. E-mail is not always effective. Many of the students waited until the Thursday evening before the next 
class (Friday evening) to read and respond to the question of the week, leaving little time for thoughtful reflection 
and reaction from their peers. Some of the strategies we tried include: 

• We created small groups based on geographic proximity. This enabled them to get together between class 
sessions. However, one group was composed of people in rural locations. They were not able to get together. 

• We gave up class time for team meeting time. 
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• We worked on class norms for e-mail discussions (short, single-topic responses, earlier and more frequent 

reading of e-mail). In fact, we used this strategy a great deal. When we encountered problems in the class, we asked 
the students to verbalize those problems and then to develop solutions that they would all agree to follow. 

“It’s Reflection Stupid” or Projects vs. Problems 

One of the things that we as teacher-facilitators learned that probably had little effect on our students is that 
there is a difference between projects and problems. In general, a problem is more open-ended than a project, leaving 
almost all decisions about where to go with the problem to the students. A project is more narrowly defined and may 
specify the nature of the final product or solution. Part of this dilemma occurred because our students felt a need to 
have everything spelled out, turning what was intended to be a problem into a project - it takes time to develop the 
trust among students and between students and teachers necessary to accept the notion that multiple solutions are 
OK. We felt kind of guilty about this at first, thinking we were failing in our approach. But we came to the 
conclusion that sometimes a project is more appropriate than a problem, especially if you are trying to teach specific 
skills that a problem would allow the students to avoid. The purpose of both a problem and a project should be 
learning through reflection. We believe there is a role for each as long as reflection is not short-circuited. Two of our 
most successful examples from each category follow: 

Example Problem : (We took this problem from the newspaper.) Your school board has decided to suspend all 
spending on technology until evidence is produced that technology has a positive effect on learning. You are 
teachers in the district. Your task is to prepare a 20 minute presentation supporting or arguing against the school 
board’s decision. 

Example Project : (We took this from student needs. They became extremely interested in student-centered learning 
strategies and wanted to learn more about them.) We assigned each of six groups one student-centered strategy. 
They researched the strategy and created a HyperStudio stack about the strategy that they other groups used. 
Strategies included: PBL, reciprocal teaching, integrated learning, cognitive apprenticeship, authentic instruction. 

Overwhelming 

Our notion of doing the program with intensive weeks in large blocks was appealing to our students. Yet, 
it took us, as teachers, half the program to figure out how to deal with it. For example, we thought that a class that 
was 6-credit hours long should deal with at least three problems/projects. And, in our effort to appear organized, we 
prepared the problems ahead of time and presented them all at the first class. It was difficult enough for us to 
organize 15 hours of class per weekend and then try to subdivide that into time for three separate projects and any 
other material we wanted to deal with. The students had the same organizational problem, not knowing how to divide 
their time away from class. They were always overwhelmed. They also tended to want to put in much more time 
than we wanted them to for each project. 

Assessment 

Assessment and grading continue to be a problem. As teachers, we constantly wresde with the conundrum 
of not believing in grades, but since the university requires them we believe they should mean something. This 
becomes a greater problem in a problem/project environment based on small groups where everything is always in 
process and feedback is supportive rather than judgmental. 

Cooperative learning and PBL experts recommend both individual and group accountability. Yet, in the area 
of group accountability we found that our students continue to be reluctant to comment on the participation of others 
and avoid evaluating their group members. We tried face-to-face meetings with the team, personal forms and 
anonymous forms, both group grades and individual grades. We were satisfied with nothing we tried. There may be 
two contributing factors to this: first, our students quickly developed such close relationships that they worked hard 
to support each other and would not let each other down. In fact, this was a cohort group and they knew they were 
going to need to work with each other for two years. Second, we took considerable time to prepare them for working 
in teams. This preparation included the following activities: 

In our first class, we asked students to write down five things that they dislike about working in small 
groups. Our students are graduate students and they’ve been exposed to small group work many times in class and at 
work. Generally students dislike it. 

We then asked students to share their lists. It doesn’t take long before you develop a list of about 10 items 
that take into consideration almost all complaints. While developing the list, with each student complaint, we then 
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ask for ideas on how to deal with the complain so that it does not become a problem. So, we ended up with a list of 
about 10 problems and several solutions. Some of the common problems include: 

• What do we do with the non-participant? 

• How do we handle the person who has a story about everything? 

• What will we do when someone is not pulling their weight? 

• How may hours a week can we each give to this project? 

• What do we do if someone doesn’t perform the work assigned? 

• How do we handle the negative person? 

• We then created the first teams, attempting to balance personalities and geographic locations. 

Each team then took a worksheet based on the complaints about group work and developed a plan for 
dealing with each problem. We made it clear to each team that communication and work problems were to be dealt 
with by the team, not the teachers. We found that the very process of discussing potential problems and creating 
team policies for dealing with the problems prevented almost all future problems. Expectations and standards were 
set and each team member signed-off on the policies. 

Despite the progress made during the first semester, we found that it was important to continue reviewing 
our standards and basic PBL principals. Several of the students, even though they intellectually agreed with the 
structure of the class, continue to push for more traditional structure and guidelines. 

Lessons Learned 

As we plan for the next cohort, there are a few decisions we have made, based on what we have learned from 
the first cohort. 

• Both students and teachers are better at dealing with one problem or project at a time. Therefore: 

• The courses for our next cohort will be arranged around problems or projects. 

• Rather than traditional 3- or 6-hour courses, we will assign credit hours based on the needs of the problem - 
anywhere from one to four. 

• Courses will not follow the university calendar, but follow one after another, so the students are only 
taking one class/problem/project at a time. 

• As we plan the courses, we will make a decision as to whether a problem or a project is more appropriate 
for the skills we are targeting. 

• Shorten the length of the weekend meetings. By Sunday morning we were all washed up. We had some 
fruitful Sundays, but not always. 

• Target a student population that is closer to the university, making between-class meetings more practical. 
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Statement of Problem 

In a climate of rapid change, increasing innovation, and proliferating knowledge, lifelong learning is an 
important educational objective. In order to keep current in their fields, people have to be willing and able to 
continually “retool” their knowledge and skill base. The need to be a continuous learner is especially apparent in the 
domains of medicine, law, business, engineering, and information technology because of the overwhelming 
explosion of information and technological advances in those fields. Nash (1994) reports: 

• More than 6,000 scientific journal articles are written every day; 

• Scientific and technical information currently increases 13 percent a year which means that this information 
doubles every 5.5 years; 

• The rate of increase will soon jump to 40 percent per year due to the increasingly powerful information 
systems and the increasing population of scientists; and 

• These increases will cause the scientific database to double every 20 months. 

Because of the exponential growth rate of information, knowledge and skills become obsolete before 
acquisition, let alone mastery, is possible. To effectively address the impact of the information explosion on the 
preparation of students for the future, professional schools and educators need to utilize instructional methodologies 
that not only help students acquire content knowledge and develop problem-solving and reasoning skills, but also 
develop lifelong learning skills. 

Importance of Lifelong Learning 

The knowledge explosion requires professionals to engage in lifelong learning if they intend to stay current 
— let alone evolve, advance, and remain competitive — in their profession. Therefore, lifelong- learning skill 
development is imperative if practitioners are expected to learn over the full expanse of their professional lives. 
Unfortunately, some of the practitioners that most need lifelong learning skills — those with careers in ill- 
structured, complex professions — are not developing them during their formal education. Regarding the lack of 
lifelong-learning skill development in schools, Walton and Matthews (1989, p. 551) state, “Some doctors from 
medical schools with the usual type of curriculum behave as if they had been immunized against further learning, and 
many doctors often do not continue to learn sufficiently.” Supporting this, a study examining doctors’ performance 
on a recertification examination over a number of years found that their performance on questions related to changes 
and innovations in their fields declined with each passing year; this decline was attributed to the doctors’ inability to 
acquire new knowledge, not as a result of forgetting previously acquired knowledge (Day, Norcini, Webster, Viner, & 
Chirico, 1988). 

“We teach most effectively when we help our students learn how to learn.. .not what to think and make and 
do in [the current year]; but how to think and how to learn for those years of life and profession than lie ahead” 
(Nash, 1994, p. 789). To achieve this requires moving away from a view of learning that is controlled outside the 
individual to a view of learning that is internally controlled by the individual (Overly, McQuigg, Silvemail, & 
Coppedge, 1980). Specifically, the ability to engage in lifelong learning is based on the development, and 
subsequent successful application, of two skill areas: metacognition and self-directedness. 

Metacognition 

Von Wright (1992, p. 64) defines metacognitive skills as “the steps that people take to regulate and modify 
the progress of their cognitive activity: to learn such skills is to acquire procedures which regulate cognitive 
processes.” Glaser (1984) describes metacognitive or self-regulatory skills as knowing what one knows and does not 
know, predicting outcomes, planning ahead, efficiently apportioning time and cognitive resources, and monitoring 
one’s efforts to solve a problem or learn. Metacognitive skills include taking conscious control of learning, 



planning and selecting strategies, monitoring the progress of learning, correcting errors, analyzing the effectiveness 
of learning strategies, and changing learning behaviors and strategies when necessary (Ridley, Schultz, Glanz, & 
Weinstein, 1992). Because metacognition involves these self-regulatory skills, it can have a positive impact on 
problem solving ability and the transfer of knowledge across domains and tasks if developed during instruction 
(Bereiter & Scardamalia, 1985; Bransford, Sherwood, Vye, & Rieser, 1986). In fact, if not developed, students have 
difficulty recognizing when they have failed to adequately meet learning goals or complete tasks (Bransford et al., 
1986). Since these are skills utilized by successful practitioners and experts (Chi, Feltovich, & Glaser, 1981; 
Bransford et al., 1986), adequately developed metacognitive ability is needed in order to engage in effective problem 
solving and reasoning activities. 

Self-direcfedness 

To be successful, students must develop the self directed learning skills needed [within the domain]. They 
must be able to develop strategies for identifying learning issues and locating, evaluating, and learning from 
resources relevant to that issue. (Savery & Duffy, 1995, p. 143) 

When dealing with real patients, the doctor has to begin assessing the patient’s condition before having all 
of the data necessary to evaluate, diagnose, and treat the patient. Characteristically, the patient provides the doctor 
with fragments of information (“My stomach hurts. I can’t hold any food down. No one else in my family is 
experiencing any problems.”). The rest of the information needed to solve the patient’s problem comes from the 
study of a variety of other resources: patient and family history, laboratory results, x-rays, other doctors’ opinions, 
past experiences, similar cases in the case file, and current research findings on new diagnostic and treatment 
procedures. The doctor has to determine what information is needed, what resource should be used to acquire the 
information needed, how to use the resource effectively, how to come to terms with opposing or contradictory 
information, and how to apply the information acquired to the problem to achieve a solution for the patient. These 
skills are described as “self-directed learning skills” (Barrows, 1985, 1986). Barrows (1995) defines the process of 
self-directed learning as utilizing the following skills to solve a problem or fulfill a learning requirement: 

• the ability to identify and define a problem/ learning need; 

• the ability to identify, find, use, and critique resources for solving the problem or meeting the learning 
requirement; 

• the ability to capture and apply information from resources to the problem or learning need; and 

• the ability to critique information, skills, and processes used to solve the problem or meet the learning 
requirement. 

In summary, lifelong learning skills, specifically metacognitive and self-directed learning skills, need to be 
developed if educators intend for their students to stay current in their fields. Staying abreast of new innovations, 
research, techniques, and information is a prerequisite for successful decision-making and problem-solving on-the- 
job. There are a number of instructional methodologies that purport to nurture the development of lifelong learning 
skills by engaging students in authentic, problem-centered learning experiences in which they are responsible for 
making decisions across all phases of the activity. The next section summarizes four of those methodologies. 

Insfrucfional Methodologies that Develop Lifelong Learning Skills 

This section examines four instructional methodologies that help students develop the metacognitive and 
self-directed learning skills needed to be lifelong learners. With common theoretical roots in cognitive psychology, 
these methodologies are: problem-based learning (PBL), intentional learning, reciprocal teaching, and cognitive 
apprenticeship. Using similar instructional strategies, these methodologies engage students in the types of cognitive 
activity needed to build knowledge, including lifelong learning skills, that students can transfer to new situations. 

Problem-based Learning (PBL) 

Problem-based learning (PBL) is an instructional methodology that uses an authentic problem, need, or 
challenge as a context for students to learn problem-solving and lifelong learning skills and acquire knowledge about 
a particular domain (Barrows & Tamblyn, 1980; Boud, 1985). 
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The basic outline of the problem-based learning process is: encountering the problem first, problem-solving 
with. ..reasoning skills and identifying learning needs in an interactive process, self study, applying newly gained 
knowledge to the problem, and summarizing what has been learned. (Barrows, 1985, p. 15) 

In a PBL environment, students work with problems in a manner that fosters reasoning and knowledge 
application appropriate to their levels of learning. In the process of working on the problem and with their peers, 
students identify areas of learning to guide their own individualized study. The skills and knowledge acquired through 
individualized study are applied back to the problem to evaluate the effectiveness of learning and to reinforce learning. 
The learning that has occurred in work with the problem and in individualized study is summarized and integrated 
into the student’s existing knowledge structure. 

PBL Strategies for Lifelong Learning. 

The development of lifelong learning skills is an important educational objective of problem-based learning 
(PBL). PBL works towards this objective during each phase of the PBL process. In phase 1 — referred to by 
Barrows (1985, p. 62) as “reasoning through the problem and identifying educational needs in counterpoint” — 
students reason aloud through the presented problem, defining what they know and do not know, formulating 
hypotheses, clarifying understanding through negotiation, critiquing peers’ comments about the problem, and setting 
educational goals and creating action plans to meet those goals. With tutor coaching and scaffolding, these activities 
help students develop the self-monitoring skills necessary to identify learning needs by making their internal 
thinking processes overt. The development of self-monitoring skills is an important part of being metacognitive 
and, therefore, contributes to students’ ability to be lifelong learners. 

During the self-directed study phase — phase 2 — of a PBL activity, students carry out their action plans 
by engaging in self-study. Students determine how long it will take to address an action plan item, create a timeline, 
and determine the required resources. This process helps students develop self-directed learning skills which is a 
critical component of lifelong learning. 

During phase 3, when students apply the information acquired during self-study to the problem, students 
critique the resources used during self-study as well as their personal research methods. By critiquing the resources 
and their research methods, students acquire insight as to what resources were helpful and why, what research 
methods were productive and why, and what resources and methods did not work, why, and how they can be 
improved for the future. Constant assessment of information sources and personal processes is critical for lifelong 
learning. 

The final phase — summary and integration of learning — is an important phase in PBL; if phase 4 is 
skipped or cut short, then the full impact of students’ PBL experience is lost. During this phase, students 
summarize what they have learned and discuss how it will be used during future problems. Students consciously 
recall and reflect on the learning that occurred while they were solving the problem, elaborate on that learning, and 
integrate it into their existing knowledge structures (Barrows, 1985). Because it focuses students’ attention on their 
learning processes, this activity further builds the metacognitive skills needed for lifelong learning. 

According to Barrows (1985), this educational cycle develops students’ lifelong learning skills. The PBL 
process engages students in activities that (1) make their thinking processes overt so they can monitor and assess the 
effectiveness of their problem analysis, reasoning skills, and knowledge acquisition decisions and processes and (2) 
encourage and enable them to assume more and more responsibility for their own instruction and learning (Barrows, 
1985; Bridges, 1992). 

Intentional Learning 

Intentional learning is learning that is actively pursued by and controlled by the learner (Resnick, 1989). 
Palincsar and Klenk,( 1992) describe intentional learning as an achievement resulting from the learner’s purposeful, 
effortful, self-regulated, and active engagement. It refers to the “cognitive processes that have learning as a goal 
rather than an incidental outcome” (Bereiter & Scardamalia, 1989, p. 363). According to Bereiter and Scardamalia 
(1989), conventional classroom instruction treats learning as an activity rather than a goal. Although learning 
occurs, it is not the higher-order learning needed for students to really understand what they have learned so they can 
apply it in the future. 

By encouraging students to take “an intentional stance toward cognition” (Scardamalia & Bereiter, 1991, p. 
37), intentional learning helps students learn how to not only monitor and be aware of their own thinking and 





learning processes (i.e., metacognitive skills), but also to take responsibility for pursuing individually-determined 
learning goals (i.e., self-directed learning) (Brown & Palincsar, 1989). 

Intentional Learning Strategies for Lifelong Learning. 

The objective of an intentional learning environment is to create a supportive structure in which students 
can engage in cooperative knowledge building as they move towards greater autonomy. Addressing students’ need for 
higher-order abilities in thinking and learning, intentional learning helps students develop the general metacognitive 
and self-directed learning skills that facilitate autonomous lifelong learning .(Palincsar, 1990; Scardamalia, Bereiter, 
McLean, Swallow, & Woodruff, 1989), specifically the ability to: 

• monitor and assess how they learn, think, and solve problems, and make adjustments when necessary; 

• make maximum use of existing knowledge; 

• ask questions to identify knowledge deficits and set personal learning goals to address those deficits in positive 
ways; 

• utilize learning strategies other than rehearsal to attain learning goals; 

• access, apply, and evaluate appropriate resources, including peers and teachers; and 

• manage the learning process (set goals, create action plans, set deadlines, identify appropriate learning 
strategies). 

These skills are developed by engaging students in situations in which they need to build a body of 
knowledge based on their learning interests and needs using a variety of information resources. While building the 
knowledge base, students practice tactics for making claims, collecting evidence in support of their claims, and 
evaluating and responding to counterarguments from peers and teachers. Through this knowledge-building process, 
students reflect on specific aspects of their learning and thinking processes, and consider the effects of collaboration 
on each other’s learning, such as the impact of opinion, bias, controversy, debate, and negotiation (Glaser, 1991). 

In addition, intentional learning prepares students for self-directed learning activities by helping them learn how to 
ask questions based on personal knowledge deficits and formulate learning goals to address those deficits. Research 
by Scardamalia and Bereiter (1991) indicates that students can learn to ask questions to guide their knowledge 
building, thus assuming more control and ownership over their learning activities. Because intentional learning 
* emphasizes question generation to guide goal attainment, students acquire ownership over learning activities, find 
personal relevance during learning activities, and develop skills needed to be lifelong learners. 

In summary, an intentional learning environment helps students to be more intentional, and therefore more 
metacognitive and self-directed, by utilizing instructional tactics that promote student autonomy. 

Reciprocal Teaching 

Reciprocal teaching is a teacher-led, cooperative learning methodology developed to improve students’ 
understanding of complex text (Brown & Palincsar, 1989; Palincsar & Brown, 1984). Reciprocal teaching provides 
a collaborative support structure “within which novices could take on greater responsibility for more expert roles” 
(Brown & Palincsar, 1989). Palincsar and Klenk (1992, p. 213) describe reciprocal teaching as: 

...an instructional procedure that takes place in a collaborative learning group and features guided practice in 

the flexible application of four concrete strategies to the task of text comprehension: questioning, 

summarizing, clarifying, and predicting. The teacher and group of students take turns leading discussions 

regarding the content of the text they are jointly attempting to understand. 

Reciprocal teaching helps students develop the comprehension-monitoring and comprehension-fostering 
skills (i.e., metacognitive skills) needed to improve reading comprehension (Slavin, 1994). Teachers model reading 
comprehension skills and students practice these skills during cooperative discussions that focus the student’s 
attention on understanding both (1) the text’s content and (2) the reading comprehension strategies being practicing. 

Reciprocal Teaching Strategies for Lifelong Learning. 

In a reciprocal teaching environment the teacher models learning strategies in a problem context that is 
shared directly and immediately with students. This makes explicit the cognitive processes that have to occur in 
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order to comprehend text; the modeling process decomposes the reading comprehension task so students can see how 
the strategies work and how to go about applying the strategies themselves (Collins, Brown, & Holum, 1991). The 
teacher’s modeling of reading comprehension skills encourages students to focus their attention on their observations 
of the teacher and then to reflect on their own performance as compared to the teacher’s performance (Collins, Brown, 
& Newman, 1989). 

The reciprocal teaching process begins with the class reading an excerpt from a text. After the excerpt is 
read, the teacher demonstrates the reading comprehension skills by articulating the questions that need to be asked 
about the excerpt in order to clarify understanding, summarizing the text, making predictions about what will happen 
next in the text, and discussing what parts of the excerpt were personally problematic. During this phase, students 
listen to the teacher in the context of knowing that they will have to undertake the same task. This focuses their 
attention on how the teacher’s activities relate to the excerpt. 

Then, with the teacher scaffolding student performance, each student takes on the role of the teacher for a 
new text excerpt, modeling the reading comprehension strategies for peers the same way the teacher did and guiding 
the direction of the group’s discussion. Throughout, other students respond to the questions, raise their own 
questions, and, in cases of disagreement or confusion, reread the text. 

Because these activities take place in a collaborative environment, immediate feedback is provided by peers 
and the teacher, helping students to (1) clarify their understanding of the text and (2) effectively utilize the reading 
comprehension strategies. 

The collaborative structure also helps students form a new conceptual model of the task of reading. By 
articulating their understanding and critiquing others’ understanding, students experience reading as a process that 
involves reflection and prediction rather than just the recitation of words. They learn to relate what they are reading to 
their needs, monitor their progress, and strive for understanding. This process of reflection and articulation within a 
collaborative group structure makes students’ thinking observable, enabling students to analyze, critique, understand, 
and improve their reading comprehension strategies (Glaser, 1991). This kind of self-awareness is critical to lifelong 
learning activities (Collins & Brown, 1988). 

In summary, with reciprocal teaching students generate their own learning goals, do their own activating of 
prior knowledge, ask their own questions, direct their own learning activity, and do their own comprehension 
monitoring (Brown & Palincsar, 1989; Palincsar & Brown, 1984). By teaching students how to flexibly apply the 
metacognitive and self-directed learning strategies of questioning, summarizing, clarifying, and predicting, reciprocal 
teaching develops skills integral to lifelong learning. 

Cognitive Apprenticeship 

Cognitive apprenticeship is an instructional methodology that supports the acquisition, development, and 
use of domain- specific cognitive tools by engaging students in authentic domain activities (Brown & Palincsar, 
1989). Modeled after the traditional apprenticeship way of learning arts and crafts, cognitive apprenticeship makes 
the usually invisible cognitive processes of an activity visible so they can be observed, practiced, and mastered with 
help from the teacher and other students (Collins et al., 1989). In order to address the problems of inert knowledge, 
learning activities are embedded in a variety of authentic contexts, creating “a rich web of memorable associations 
between them [concepts and facts] and problem-solving concepts” (Collins et al., 1989, p. 457). Cognitive 
apprenticeship focuses on “the leaming-through-guided-experience [of] cognitive and metacognitive, rather than 
physical, skills and processes” (Collins et al., 1989, p. 457). 

Cognitive apprenticeship teaches students the processes that experts use when addressing complex tasks 
within a professional domain. To this end, cognitive apprenticeship activities create a learning environment in 
which students not only acquire the conceptual and factual knowledge of a domain, but also the problem-solving 
strategies and heuristics, control strategies (metacognitive skills), and learning strategies (self-directed learning skills) 
of the domain (Collins et al., 1989). Therefore, just as the format of traditional apprenticeship helps an apprentice 
learn how to use the tools of the trade, cognitive apprenticeship helps students learn how to use the cognitive tools 
required for lifelong learning by enabling students to acquire, develop, and use cognitive and metacognitive tools as 
they engage in authentic domain activity (Brown & Palincsar, 1989). 

Cognitive Apprenticeship Strategies for Lifelong Learning 

To help students develop metacognitive and self-directed learning skills, cognitive apprenticeship employs: 
modeling, coaching, scaffolding and fading, articulation, reflection, and exploration (Collins et al., 1989; Collins et 
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al., 1991). The first three strategies — modeling, coaching, and scaffolding/fading — help students develop 
cognitive and metacognitive skills via observation and guided, supported practice. Articulation and reflection help 
students focus on their problem solving processes through elaborative activity in order to acquire conscious access to 
and control over their own metacognitive and self directed learning processes and strategies (Collins et al., 1989). 

Exploration encourages learner responsibility and autonomy in solving problems. 

In a cognitive apprenticeship environment, these six instructional strategies make cognitive processes that are 
normally invisible visible, just as in a traditional apprenticeship a tailor makes sewing processes visible to an 
apprentice. For example, a teacher may first model a cognitive task by thinking aloud while performing it, making 
the thinking processes visible to students. Then the teacher watches, coaches, and provides feedback and support as 
students practice portions of the task. As students demonstrate their ability to complete tasks on their own, the 
teacher gradually removes supports, turning over more and more responsibility for the learning process to students. 

Cognitive apprenticeship also encourages students to articulate and reflect on activities so they can elaborate 
on the learning that has occurred during the modeling-coaching-scaffolding/fading cycle. For the same reasons the 
review phase is used in PBL, articulation and reflection activities make tacit cognitive activities overt and observable 
so students can assess what they have learned, what worked and did not work, and what they will do differently in the 
future. 

Finally, when supports have been faded and students are able to monitor and assess their own learning and 
thinking processes, exploration is possible. Students need to engage in exploratory activities to learn how to engage 
in problem-solving activities, from start to finish, on their own (Collins et al., 1989). In other words, learning how 
to engage in exploration enables students to be self directed learners. 

Throughout this process, cognitive apprenticeship helps students develop the skills needed to be 
autonomous learners by engaging them in the activities of the authentic domain: the domain in which the skills and 
knowledge they are learning is applied. Part of the authentic domain that cognitive apprenticeship activities reflect is 
the social context in which learning occurs. “Learning, both outside and inside school, advances through 
collaborative social interaction and the social construction of knowledge” (Brown & Palincsar, 1989, p. 40). In a 
cognitive apprenticeship environment, collaboration provides students with an additional source of scaffolding during 
learning and problem-solving activities. Collaborative learning and problem-solving requires students to share their 
knowledge and skills, giving them additional opportunities to clarify understanding and assess overall processes; 
sharing knowledge and skills with others fosters the situated articulation of processes and concepts, helping students 
acquire conscious access to and control of cognitive and metacognitive processes (Collins et al., 1989). 

Commonalties Across Methodologies 

The instructional methodologies described in the preceding section create knowledge-building communities 
that help students develop lifelong learning skills. Interestingly, these methodologies employ similar instructional 
strategies to prepare students for lifelong learning, specifically: 

• collaboration, 

• reflection, 

• student autonomy activities, and 

• intrinsically motivating activities. 

This section summarizes how these methodologies use these strategies to fulfill the lifelong learning objective. 

Collaboration 

Instructional methodologies focusing on the development of lifelong learning skills employ collaboration 
to promote thinking because collaborative activities engage students in an interactive approach to learning (Johnson 
& Johnson, 1986). The methodologies described in this paper recognize the importance of collaboration, social 
cognition, and the social context of learning. The social context — manifested in collaborative group activities — 
elevates thinking, learning, and problem-solving to an observable status (Glaser, 1991), making students’ 
metacognitive processes apparent. This provides students with opportunities for understanding and sharing these 
processes — refining, strengthening, and extending their metacognitive skills (Von Wright, 1992). 

Collaboration and using peers as resources plays an important role in problem-based learning. First, PBL 
employs collaboration to provide students with opportunities to see and hear how other students approach and solve 
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problems. Because the students in a collaborative group are working closely together, students are able to share ideas 
and perspectives, as well as help each other clarify issues. This is important in helping students develop 
metacognitive skills. 

Second, PBL problems are complex because they are authentic. Students may not be used to or able to 
tackle a realistic problem on their own. Students working together collaboratively can often successfully tackle 
problems that individual students working alone would not be able to handle; collaborative learning can ’’give rise 
synergistically to insights and solutions that would not come about without them [the members of the collaborative 
group]” (Brown & Palincsar, 1989, p. 40). 

Finally, PBL encourages articulation through collaboration. During the problem analysis phase of PBL, 
students describe what they know and do not know about a problem and what they need to learn. When preparing for 
self-study, students determine how long it will take to fulfill a learning goal, what strategies they will employ, and 
what resources they will access. Being able to determine learning needs and plan a method of attack are important 
metacognitive skills for lifelong learners. Because students are working collaboratively during problem solving, 
their thinking processes — or metacognitive skills — are observable and therefore open for personal and peer 
assessment and refinement. Accordingly, collaboration provides students with a support structure while they develop 
the metacognitive skills needed to be lifelong learners. 

In intentional learning environments, students develop higher-order thinking skills by working together to 
build knowledge bases. Collaborative knowledge building requires that students articulate what they have learned, 
what questions are left unanswered, and their plans for future learning. These activities force students to think 
metacognitively. Also, because students are pursuing individual learning goals, the collaborative structure provides 
students with guidance and coaching, through collaborative critiquing, while they are acquiring self-directed learning 
skills. Therefore, within an intentional learning environment, collaboration is used to provide support for students 
as they develop the skills needed to be lifelong learners. 

The group discussion aspect of a reciprocal teaching activity is like the group problem analysis phase in a 
PBL activity. During group discussion, students review content (summarize), attempt to resolve misunderstandings 
(clarify), anticipate possible future text development (predict), and assess what they have learned from the text 
(question). These activities encourage students to articulate and bring into the open the internal dialogue that 
experienced students engage in to foster comprehension. The reciprocal teaching procedure helps students learn how 
to engage in that internal dialogue by making them think aloud. Because students are thinking aloud, peers and 
teachers can provide guidance and support until the reading comprehension strategies are incorporated into students’ 
repertoire of learning strategies (Brown & Palincsar, 1989). Reliance on collaboration in reciprocal teaching is 
critical to the development of the metacognitive skills needed to learn from expository text — the same 
metacognitive skills needed to engage in lifelong learning activities. 

In a cognitive apprenticeship environment, collaboration provides additional scaffolding while students learn 
cognitive and metacognitive skills (Collins et al., 1989). Because students share knowledge during collaboration, 
they must articulate what they know. If misunderstanding occurs, students clarify points and assess their overall 
understanding. This forces students to develop metacognitive awareness, a key component of the lifelong learning 
construct. Students involved in cognitive apprenticeship activities develop critical metacognitive skills that will 
enable them to perform as lifelong learners. 

Reflection 

These instructional methodologies also encourage students to review and reflect on what they have learned 
and how they have learned. Self-reflection activities are embedded into instructional activities to support the 
development of metacognitive skills. “Self-reflection implies observing and putting an interpretation on one’s own 
actions, for instance, considering one’s own intentions and motives as objects of thought” (Von Wright, 1992, p. 
61). Von Wright describes the process of self-reflection as the ability to think about one’s self as an intentional 
subject of personal actions and to consider the consequences and efficacy of those actions. This involves the ability 
to look at one’s self in an objective way and to consider ways of changing to improve performance. 

Reflective skills play an important role in the development of metacognitive skills. Dewey (1933) 
described the value of reflection as a component of educated thinking. Contemporary researchers have further 
elaborated on this argument by describing the relationships between reflection, metacognitive skills, and the 
development of problem-solving strategies (e.g., Kuhn, 1989). The specific contribution of reflection in the 



development of metacognitive skills is its role in consolidating the development of new strategies and encouraging 
transfer (Schauble, Raghavan, & Glaser, 1993). 

Even though reflective activity is important, it is possible for students to be so caught up in completing a 
task that they fail to reflect, hindering what they learn. “We can keep students so busy that they rarely have time to 
think about what they are doing, and they may fail to become aware of their methods and options” (Wheatley, 1992, 
p. 536). Schon (1983) refers to this as being “in the action” rather than reflecting on the action. If students do not 
have opportunities to examine their methods and options, they will not develop the metacognitive skills needed for 
lifelong learning. Therefore, teachers need to support students in reflecting on their own learning and problem- 
solving processes, as well as on what they have learned (Schon, 1987). 

Blakely and Spence (1990) describe several basic reflective strategies that need to be employed by an 
instructional methodology to develop metacognitive skills: 

• Students should be asked to consciously identify what they “know” as opposed to “what they don’t know”. 

• Students should keep journals or logs in which they reflect upon their learning processes, thinking about what 
works and what does not. 

• Students should manage their own time and resources including estimating time requirements, organizing 
materials, and scheduling the procedures necessary to complete an activity. 

• Students should engage in guided self-evaluation through individual conferences and checklists to help them 
focus on the thinking process. 

Blakely and Spence’s strategies illustrate the importance of student reflection in promoting the development of 
metacognitive skills. 

Reflective activities, as defined by Blakely and Spence, are a common characteristic of the instructional 
methodologies described in this paper. A common goal of these methodologies is to induce students to include self- 
reflection as an essential component of their action strategies in the context of learning and problem-solving (Boud, 
Keogh, & Walker, 1985; Harris, 1989; Von Wright, 1992). 

In a problem-based learning environment, reflection occurs during the review phase. During this phase, 
students assess the effectiveness of their learning strategies and accomplishments. They critique the resources used, 
reexamine points of agreement and disagreement, clarify comprehension problems, and determine what they would do 
differently in the future to improve on their own personal learning process. The ability to reflect on personal 
learning strategies and processes is necessary for enhancing metacognitive ability, and preparing for new learning 
issues and challenges. 

In an intentional learning environment, the importance of students’ awareness of the functional potential of 
knowledge for the acquisition of other knowledge is emphasized (Bereiter & Scardamalia, 1989). According to 
Bereiter and Scardamalia, of equal importance to becoming an intentional learner is the awareness of the potential of 
self-reflection as a tool for engaging in intentional learning activities. Understanding the functional potential of 
knowledge for the acquisition of new knowledge is critical to lifelong learning. Part of being metacognitive, 
students must learn to use what they have learned — in terms of content and skills, as well as personal learning 
strategies and processes — to improve subsequent learning activities and further future learning endeavors. 

Besides providing social support, shared expertise, and role models, reciprocal teaching also stimulates self- 
reflection. In fact, central to the reciprocal teaching methodology is the emphasis on social interaction as a condition 
for developing students’ reflective skills (Brown & Palincsar, 1989). Reflective skills come into play when students 
take on the role of teacher and lead group discussions. When leading the group discussion, it is necessary for the 
student to decide what has to be explained or taught based on the group’s conceptions and misconceptions. In 
addition, the student has to assess how one’s teaching is impacting the group’s conceptions and misconceptions. 
Trying to understand another person’s perspective on an excerpt of text, a requirement for participation in a reciprocal 
teaching activity, forces students to reflect on their own perspectives. 

In a cognitive apprenticeship environment, reflection is encouraged so students will elaborate on what they 
have learned. Again, as in the review phase of PBL> reflection activities make tacit cognitive and metacognitive 
activities overt so students can assess and improve them. Having opportunities to reflect on metacognitive skill 
utilization is necessary if students are expected to develop these skills to the point of using them without prompting 
or guidance. Reflection enables students to assess and improve their skill use to affect a change in subsequent 
utilization practice; without reflective activities students may never change their utilization patterns, regardless of 
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effectiveness. Therefore, reflective activities help students assess how they are learning and improve their strategies 
and processes, preparing them to independently utilize appropriate metacognitive skills during lifelong learning 
activities. 

Student Autonomy 

All of these methodologies engage students in activities that gradually increase their control and 
responsibility over the learning process. Creating autonomous learners, these methodologies teach students how to 
plan their learning: how to address their learning needs, set learning objectives, employ learning strategies, utilize 
resources, and assess the overall process. In other words, these methodologies help students acquire more agency 
over their zones of proximal development. For the most part, this is done through guided social interaction. 

The tutor in a problem-based learning environment provides a supportive structure while students learn to direct their 
own learning. When students are stuck and can go no further on a problem, the tutor asks questions that help 
students refocus on the problem, enabling them to proceed. Another source of assistance is keeping a record of the 
group’s progress on the chalkboard. This activity, started during PBL’s problem analysis phase and updated 
throughout the other phases, is a method of externalizing thinking processes and helps students remember and 
organize their thoughts. 

In an intentional learning environment, students are required to build their own knowledge bases founded on 
personal learning needs and interests. To do this, students must take responsibility for managing the learning 
process. This includes setting goals, creating action plans, setting deadlines, and identifying appropriate learning 
strategies. Even though students are required to manage their own learning, they are supported by peers and teachers 
who help them move towards autonomy. A PBL environment is very similar. During phase 1 of a PBL activity, 
students identify knowledge deficits, set learning goals based on those deficits, and create action plans to fulfill those 
learning goals. During the self-study phase, students determine what resources they will use to address their action 
plan. These decisions are all made by the students. Although students are supported by each other and their teacher, 
they are responsible for managing the learning process, helping them achieve greater student autonomy over learning 
activities. 

The modeling, coaching, and scaffolding/fading process employed by reciprocal teaching and cognitive 
apprenticeship provides assistance so students can accomplish a task they would not be able to carry out without 
help (Vygotsky, 1978). Modeling is the demonstration of the problem-solving process. Through modeling, 
students obtain a complete mental picture of the process they are learning because the cognitive and metacognitive 
processes are explicit. Coaching provides students with guidance while they practice solving problems. As students 
begin to solve problems on their own, the supportive structure is slowly faded until it can be completely removed. 
This process helps students develop metacognitive awareness while preparing them to be self-directed and lifelong 
learners. 

Intrinsically-Motivating Activities 

For students to be oriented toward lifelong learning, they must be willing to continue to learn. This 
willingness to learn is a product of intrinsic and continuing motivation (Kinzie, 1990). Intrinsic motivation is the 
desire to pursue a goal in which the primary reward is the pursuit itself. When students are intrinsically motivated, 
they are more likely to be more self-determined; they may attempt more problems, focusing on the way to solve the 
problem instead of on finding the correct solution (Condry & Chambers, 1978). The self-determination and desire to 
return to a learning activity is part of lifelong learning. Therefore, motivation to engage in lifelong learning 
activities may have an impact on students’ determination and desire to engage in lifelong learning in the future. 

The instructional methodologies described in this paper employ intrinsic motivation strategies to get students excited 
about learning. Intrinsic motivation strategies are based on the idea that students will expend more effort on tasks 
and activities they find inherently enjoyable and interesting, even when there are no extrinsic incentives (Keller & 
Burkman, 1993). The instructional methodologies described in this paper employ the following tactics to encourage 
and sustain intrinsic motivation: 

• Students are actively engaged in leading, recording, discussing, facilitating, making decisions, collaborating, 
making presentations, and evaluating throughout the learning activity. 
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• Students fulfill higher-level objectives and answer divergent questions. At the heart of each learning activity 
is a problem to be solved. Solving the problem involves analyzing the situation presented, applying existing 
and new knowledge, evaluating alternative solutions, forecasting consequences, and assessing the problem 
solving process. 

• Students take ownership of the problem by assuming the roles of real stakeholders. When students take on 
the roles of scientists, programmers, historians, etc., their “motivation soars because [they] realize it’s their 
problem” (Stepien & Gallagher, 1993). 

• Students engage in authentic activity. The advantage of engaging students in authentic problem-solving 
activities is that students become much more aware of how the knowledge they are acquiring can be put to 
use, improving their ability to transfer their knowledge and skills to future problems. “Adopting a problem- 
solving mentality... reinforces the notion that the knowledge is useful for achieving particular goals. Students 
are not being asked to store information away; they see how it works in certain situations...” (Prawat, 1989, 

p. 18) 

Because intrinsic motivation impacts continuing motivation to learn, involving students in activities that 
make them excited about learning is critical not only for the success of the learning activity itself, but also to the 
development of students’ commitment to learning throughout their lives. 

Conclusion 

This paper provided an overview of instructional methodologies — problem-based learning, intentional 
learning, reciprocal teaching, and cognitive apprenticeship — that prepare students for lifelong learning. Using 
collaboration, reflection, student autonomy, and intrinsically-motivating activities, these instructional methodologies 
help students develop the metacognitive and self-directed learning skills needed to remain competitive in an ever- 
changing professional climate. 
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Abstract 

Interactive instructional television (ITV) is becoming more popular as a form of distance education. In 
order to provide the best instruction for all students ; it is important to examine both the environment in which ITV 
is used and the ways in which it is implemented. This study indicates that even as ITV is reputed to be successful 
and liberating , it is potentially obstructive and disempowering at the same time. Care must be taken to include the 
teachers and students who use instructional television in all stages of the planning } implementation , and evaluation 
processes. 

School districts are using more ITV each year, for many reasons. Those reasons range from the practical 
needs of serving students to the desire to integrate technology into the learning process. In other words, ITV can be 
driven by the curriculum need or by the seduction of technology. Interactive instructional television (ITV) can 
provide a rich and powerful learning environment. It can employ techniques of visual learning in meaningful ways. 
If implemented appropriately, ITV can allow the sharing of visual resources across various groups of people, in 
different geographical locations. 

For both rural and urban settings, ITV offers flexibility in scheduling by enabling schools to offer multiple 
sections of a course, if necessary. This can be especially important in corporate training rooms where various work 
schedules need to be accommodated in order to participate in the training. Hence, many people regard ITV as a 
practical tool that helps high schools better prepare students to succeed in higher education and within the career 
marketplace, and helps improve workplace skills by providing current training into corporate situations. 

The affordances of ITV are especially important for small, geographically isolated schools in that ITV 
provides enhanced curriculum by way of providing access to advanced or specialized courses and introducing 
flexibility in scheduling by way of multiple course sections. Hence, ITV is regarded as a practical tool by which 
schools might graduate students better prepared to succeed in higher education and better equipped to compete in the 
career marketplace. 

Politicians are pushing for increased funding for and utilization of ITV by convincing people that ITV 
increases the interaction between teachers and students. Although each situation is unique, the common thread 
among all who implement ITV is that they claim to be seeking a better way to serve various educational needs. 
Many people believe that the most important advantage of ITV for small, geographically isolated schools is that ITV 
provides access to advanced or specialized courses that would not otherwise be available to them. 

Although there is no doubt about ITV’s utility in providing enhanced curriculum opportunities and 
scheduling alternatives, the outcomes of using ITV are varied and complex. Despite the fact that ITV’s utility in 
providing curriculum and scheduling alternatives is indisputable, a void in the literature pertaining to the 
complexities of ITV’s attendant outcomes requires attention. This paper considers important aspects of ITV 
learning environments that have yet to be adequately explored. It contributes to discourse surrounding equity in 
opportunities for constituencies who are under-represented in the decision making process surrounding ITV adoption 
yet account for a great portion among faculty and students. It touches on material expression of a culture’s value 
system. It illuminates the power structures under whose aegis ITV networks operate. It reviews common design 
features of ITV classrooms and speculates upon the nature of ITV instructional presentations. The upshot of this 
preliminary work indicates that even as ITV is reputed to be successful, it is potentially obstructive toward goal 
achievement for students and instructors whose needs and preferences are ignored throughout the planning, 
implementation and evaluation processes. 

This study suggests that even as ITV is reputed to be successful and liberating, it is potentially obstructive 
and disempowering at the same time. It can indeed interfere with goal achievement for students and instructors 
whose needs and preferences are ignored throughout the planning, implementation, and evaluation processes. 
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Culture and The Power Structure 

A meaningful examination of the merit surrounding any educational practice requires viewing it within the 
realm of its culture. Institutions, technologies, policies, and practices are material expressions of the dominant 
culture’s ethos and its attendant meanings (Cockburn, 1983; Trask, 1986). Therefore an assessment of the status of 
ITV requires an unbiased inspection of the institutions involved, technologies employed, policies determined, and 
common practices within a cultural perspective. 

Educational institutions are purposeful enterprises driven by power structures that are informed by openly 
endorsed agendas of both political and personal origin (Freire, 1970; Knupfer, 1993; Shor & Freire, 1987). The 
composition of these power structures in relation to ITV is disclosed by an analysis of one Midwest state’s 
instructional leadership structure and a close examination of ITV implementation. This analysis reveals that 
leadership and decision-making is clearly dominated by males who hold nearly all of the leadership roles, while the 
people who implement the ITV are females (Kansas State Board of Education, 1995). Preliminary analysis of the 
various constituencies that guide ITV network operations yields a similar result. Preliminary analysis of the various 
constituencies that guide ITV network operations yields a similar result. These outcomes justify speculation that 
students and faculty are likely to be differentially served in ITV course delivery in ways related to the physical 
attributes and value systems of the dominant culture as expressed through a cadre of influential decision makers. 

Perhaps that is not important, then again, some reflection upon the situation raises questions that suggest 
that it might be more important than one would immediately realize. For example, consider the possible answers to 
the following questions. Are students and faculty best served by decisions that are made by the dominant, male 
administrative culture? If so, why and if not, why not? In which areas might there be different emphases, depending 
on the cultural background of the decision makers? What patterns of decision-making are evident in terms of ITV 
course delivery? To what extent do the current ITV facilities and practices support the needs of those who actually 
use them? In what ways do the current ITV facilities reflect the needs of the decision makers but fall short of those 
who implement the technology? To what degree are the physical attributes and value systems of the decision makers 
reflected in the design of ITV environments and how does this affect the instructors and learners? 

The nature of the differential service is revealed in ITV’s role as a technology; where technology, as a tool, 
is generative of its own and sometimes exclusionary language and meanings, as well as evocative of prescribed ways 
of thinking and acting (Cockburn, 1983; Lakoff, 1983; Segal, 1994; Zimmerman, 1983). ITV classroom designs 
and ITV activities are therefore material expressions of the decision makers’ values systems. The power of this 
technology and its apparent predisposition to serve selectively drags ITV squarely into discussions about visual 
communication and gender equity, especially since the manner of how communication is supported and conveyed is 
predominantly informed by males. 

From time immemorial, males have been in decision-making roles within American homes, businesses, 
and educational institutions. Society has carried messages about gender stereotyping within instructional products 
and schooling practices, through the mass media, within social practice, and so on, thus perpetuating the situation 
complete with its advantages and disadvantages. Of course, there are good points to male leadership but there also 
are drawbacks. The disadvantages are likely to occur when decisions are made in isolation without regard to the 
female perspective. 

As we witness a new time in education, when people are calling for school reform and the implementation 
of technology all in one breath, there is an opportunity to ask once again, whether or not there is adequate 
representation of the teachers, mostly female, in the business of ITV. It seems that the role of women in relation to 
the new media has made little progress, possibly even less than that within the more traditional forms of schooling. 
Could this be because technology has been viewed as male territory? 

The deeply ingrained cultural stereotypes and practices related to gender continue to support instruction in 
its historical sense by constantly feeding the old system while stifling attempts to pay serious attention to gender 
equity (McCormick, 1994; Gomick & Moran, 1972). Materials developed for use in public, private, and military 
schools as well as instructional messages delivered to the public through advertising, television, and public service 
messages continue to portray males and females in stereotypical ways, with males in dominant positions. This has 
indeed been carried forth into the gender messages portrayed to the public about using the new technologies (Knupfer, 
1996). Common practice dictates that males will serve in leadership and authority roles while females serve in 
subordinate positions. We see it in the images of males and females in the media, and we see it in practice. Thus 
decisions about instruction are often made by males and need to be implemented by females. 

The complexities of this situation are enormous. Like society itself the complexities reflect the dynamics 
of different situations in different ways, among different individuals. An examination of the complex relationship 
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between instructional planning and design, and gender reveals inequities that result from a persistent pattern of 
practice. Recognizing the result of those inequities can be easier than finding the causes and correcting the problem. 

Inequities that result from the traditional practice of male decision making and female responsibility for 
implementation within the instructional system often go unrecognized because they emerge not just as a result of 
what has been done, but also as a result of what has been left undone. The neglect and omission of the female 
population reveal themselves in subtle ways on an individual basis, but as a collective result appear throughout 
society as something that begins in the home, and perpetuates itself through schooling and employment practices. If 
that were not the case, then there would be no need for recent efforts to attract girls into fields of study that are 
typically populated by males (Kable & Meece, 1994) and the number of distressing stories about females succeeding 
despite the myriad of obstacles (Aisenberg & Harrington, 1988; Clark & Corcoran, 1986; Frenkel, 1990; Gomick, 
1990) would no longer be told. 

Innovation and ITV 

Implementation is change in action; it is the deliberate spreading of novel, specific change in a planned 
manner (Miles, 1964). The final outcome of any attempted innovation depends directly upon proper 
implementation. Fullan (1982) diagrammed four components of the evolutionary process of change and 
implementation as being along the spectrum of initiation, implementation, continuation, and as the genesis and 
proposal of an innovation. Implementation embodies the proposal’s adoption and use. Continuation involves 
complete midcourse follow-through by all participants to determine strengths and weaknesses of the implementation. 
Finally, the outcome of the change process entails the evaluation of success or failure. 

After being utilized for many years in various ways (Cuban, 1986), distance education efforts are now 
placing a new emphasis on ITV that supposedly is more accessible and affordable then in previous years. 
Unfortunately, the apparently easy access permits teacher and administrators to overlook the need for thorough 
planning in the rush to place the ITV facilities. Yet without firm direction, ITV could end up in the junkyard with 
many other poorly-implemented innovations. 

The conditions under which an innovation is implemented contribute just as much to the nature of the 
outcome as do the status of staff preparation and the presence of knowledgeable adopters. Such conditions include 
the classroom organization, the availability of instructional resources, and the more general demands of the school 
system or district. The competing demands of modem educational environments impress upon the innovator the 
need to adapt instructional media to the circumstances and with consideration of the sociopolitical climate (Knupfer, 
1993). 

The teacher’s role in school innovations is so important that educational change depends on what they do 
and think (Cuban, 1986; Fullan, 1982); Sarason, 1971). Because the teacher lives and works in a classroom with its 
own built-in imperatives and social culture, the teacher’s real working conditions must be taken into account when 
planning and implementing any educational innovation (Knupfer, 1993). The teacher will judge the acceptability of 
any innovation by its conformance to current needs and objectives, and not according to some agenda that is foreign 
to the teacher’s experience (Knupfer, 1988). Teachers must help to guide educational change, and not be its victims 
(Knupfer, 1987). 

As a part of the current emphasis on using technology in education, ITV is a fast growing and popular area 
that holds a prominent position in distance education efforts. In this position, it will have great impact on its users. 
All too often, political and fiscal forces operate beyond the teacher’s control and adversely affect change efforts 
(Fullan & Stiegelbauer, 1991; Weinshank, Trumbull & Daly, 1983). There is a tendency to oversell innovations to 
obtain funding and get them adopted by policy-makers, teachers, and others (Knupfer, 1987). Implementation of any 
change requires that the change have meaning for the teacher (Fullan & Stiegelbauer, 1991). Teachers’ concerns 
about an innovation can be abated or exacerbated, depending on how the innovation is introduced (Knupfer, 1987). 

Like any other educational innovation, the teachers must be involved in the planning and implementation of 
ITV. If teachers are to use ITV productively, then it needs to be implemented in such a way that it meets their needs 
and they are not uncomfortable. Many scholars insist that great strides have been made concerning gender equity in 
education, but it has not been enough and clearly, males are still making decisions about ITV without the input of 
the female teachers. Not only are the majority of decisions about technological purchases being decided by males, 
but also the decisions about the design of ITV classrooms and utilization of equipment as well, thus leading to 
situations that are decided by males and structured for males users. 
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ITV Features and Implications 

The various environmental features of ITV-delivered courses may differentially influence learning through 
contributions to cognitive load by way of conflicts between perceptions and expectations. Interactive Television is 
purposefully designed to optimize interaction between instructors, students and content. Therefore factors that 
influence communication potential would be revealed in the extent and depth of communication and understanding. 
On the surface this assumption seems contrary to Boak and Kirby’s (1989) observations that females express 
heightened communication apprehension in formal settings such as typify traditional class delivery and meetings yet 
score as well, or better, on exams compared to males. The contradiction is somewhat resolved by Warren’s (1989) 
and Patoine’s, (1989) conclusions that students who encounter “alien or inhospitable” learning environments tend to 
exercise one of three options: conform at a personal cost of greater effort expenditure; delay timely interaction while 
searching for alternative action aimed at attaining the same goal; or practice communication avoidance while 
grappling with an overwhelming sense of betrayal and helplessness that often portends withdrawal or failure. 

Kirby and Garrison (1989) affirm that instructional leaders and policy makers typically possess only 
superficial knowledge about educational technology. As a consequence technology consultants are sought out and 
commissioned to make material the meaning of technoculture’s exclusionary language. Because this technology is 
also dominated by males, their powerful role in guiding ITV decisions strengthens and further perpetuates the 
gendered ethos that is visible in ITV classroom designs, layout of presentation technologies, and perhaps too, 
presentations in general. Evidence of that dominion is clearly visible in all but the latter. 

Drawing upon the aforementioned studies, personal experience, and colleagues anecdotes; preliminary 
profiles of ITV classroom design features were undertaken. Analysis of this information underscores a well grounded 
hypothesis that ITV classrooms are designed according to some metric other than that of instructors’ and students’ 
needs. Overwhelmingly site designs ignore the various continua of users’ physical attributes and psychological 
needs. Evidence that gendered messages are made concrete and then replicated in the dimensions of furnishings, 
locations of audio and video devices, and types of audio handling devices are visible by degree only. 

Data collected in the various design categories of furnishings, dimensions, and layouts indicate a definite 
bias. The console dimensions and the fixed distances between device controls favor a Titan standard from a 
diminutive individual’s perspective. The height of the console’s worksurface and the distance between the work area 
and control panels compels instructors to exhibit behavior similar to that of students in inhospitable educational 
settings - conform, seek alternatives, or withdraw. 

Such attributes also convey negative messages to students. Unnatural physical relationships between the 
console components and the instructor often translate into disruptions in presentation flow that erroneously 
communicates that an instructor lacks organization or possesses weak presentation skills. Worse yet, unavoidable 
pauses attributed to design may be perceived as a lack of preparation which may suggest to some viewers that they 
rank quite low on the presenter’s list of priorities. 

Adding insult to injury, for some instructors the camera and monitor banks are located such that the 
presenter is obstructed in maintaining the illusion of eye contact with all viewers. Another negative message is 
sent. Students messages are missed. Yet another erroneous interpretation is formed. 

Unfortunately, problems with camera locations are not restricted to simulated presence. The potential for 
camera placement to utterly disable an instructor was revealed by an instructor whose camera tracking device failed 
causing the camera to be fixed on her ample bust. Hijacked by her emotions, armed with nominal technical training, 
and her novice status congealed rendering her unable to marshal her sensibilities toward a solution free from the 
leering gaze of the camera. She canceled class and no longer enters the ITV classroom with confidence. The camera 
is now her nemesis. Teaching another ITV course is unlikely. What is more likely is that female students across 
the sites were similarly embarrassed by that powerful image and her response to it. 

Viewers are thus not immune to the various design limitations of ITV classrooms. Beyond that already 
showcased, the gendered perspectives of those who inform the decision-making process afflict students through 
selections in furniture designed for some generic body type. Furniture selected solely for its durability, cost, and 
purported universality can quickly translate into an inhospitable learning environment as furnishings are typically 
not selected to suit the needs of students whose physical attributes are at the extremes of various continua. Students 
quickly ascertain the likelihood of physical comfort in a learning environment and perhaps too, the attendant success 
potential. Just as a classroom that is too hot or cold, too dark or bright, or too noisy contributes to a student’s 
burdens so, seemingly, would personal comfort. 

Placement of desktop microphones further exemplify the authority of a gendered metric in ITV classroom 
design. Microphone spacing and elevation may signify that microphones are meant to be shared or are perhaps 
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objects for which students shall openly compete. It may signify that one must invade another’s space in order to 
successfully enter into a dialog. More potentially negative messages take form. Institutional ignorance of 
proxemics may be as much an obstruction to communication and intellectual involvement as are displays of 
dominance by boorish students who stake out turf in front of a microphone and silently dare others to intrude. 
Ignorance is such a poor defense when information about needs was not solicited from those expected to use the 
technologies. 

Microphone elevation appears to be yet another issue left unconsidered during the ITV planning process. 
Pedestals that place the microphone beyond the cone of effective signal generation yield weak signals. The speaker 
may thus seem uncertain or timid. Corrective actions toward achieving a stronger signal such as speaking louder or 
repositioning oneself relative to the microphone may be undesirable or impossible for the student. 

In any case, initial perceptions about the utility and hospitality of an ITV classroom may be drawn from 
interpretations about the meanings of furnishings and room layout . First impressions may potentially set the tone 
for later interactions and general expectations. Who shall be invited into the interaction and who shall be merely 
tolerated? Whose expectations should be wholly indulged and whose superficially humored? The key informers of 
ITV designs are primarily represented by males. Is that dominion manifest in the ITV classroom design? 

This bias is evident, by degree, in the physical attributes of the ITV sites examined. Manifest in classroom 
objects it is reasonable to suspect that it should also be evident in class presentations. Preliminary observations 
affirm that the influence of the dominant culture’s perspective is discernible in the graphic art and content of 
presentations though diffident in presentation methods. 

Everywhere observable or not, the most insensitive display observed occuned at a network dedication 
ceremony during which a psychology instructor opened his presentation with a video-tape image of his disguised self 
intensely examining a Playboy magazine centerfold from various angles. The women in attendance shook their 
heads in displeasure while a great number of males laughed and grinned. Other men, a small number, did not visibly 
respond. Follow-up conversations with several of the women, without exception, deemed the presenter to be a 
consummate boor, ignorant of the boundaries of good taste, and definitely the rube he portrayed. The other men were 
judged as condoning and encouraging repetitive events because of their laughter. Those males who did not respond 
were judged as either ignorant that this display was offensive or were offended as well. While this may be an isolated 
event the value system that drove its production and allowed it to pass muster can be legitimately isolated as a driver 
of media content and selection under general circumstances. 

A parallel study of instructional presentations, warranted on the basis of observations and others’ anecdotes, 
substantiates a belief that the dominant culture’s value system is permissive both in portraying others in disparaging 
ways and in omitting portrayal of other cultures. Observations of presentations prepared by workshop attendees are 
consistent with suspicions that even when schooled in the rudiments of graphic design for ITV presentations, 
instructors are marginally attentive to the total visual message. 

It would be a mistake to make an unfounded leap toward assigning apparent gender bias in presentations to 
cultural dominance since other factors contribute substantially to presentation content and delivery. Among those 
factors are the 1) amount of time available for production of presentation slides, 2) lack of available scanning 
technology, 3) nominal skills in production design and development^) ignorance of issues in visual literacy, and 
finally, 5) limited clipart libraries bundled with presentation software. These circumstances combined with cultural 
values promote the use of canned graphics on prepared templates in ways that yield a visual concoction that 
transmits conflicting and negative messages. 

Recommendation 

Though standardization across member sites of any particular network is a desirable feature great care must 
be taken by decision makers to insure that the many needs of users are identified and equitably addressed. This means 
inviting and embracing the participation of various representative constituencies into fact finding and decision 
making processes. Acknowledging that not all needs can be immediately met, it is important to view ITV 
classroom designs with amenability to adaptation as a guidepost. It also means deliberate funding of line items 
aimed at a spectrum of planning check points from current to long range toward meeting users’ needs. In doing so, 
perhaps the burdens students bear will be lightened in some measurable way. 
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Human Performance Technology - HPT - is a rapidly changing field; and the theories that inform ID are 
insufficient for HPT As a field currently rooted more in application than in research HPT would benefit by being 
proactive in its research agenda, rather than reactive to external forces. This proactivity is best based in grounded 
theory. A thoughtful and cogent theory-building process is preferable to the current piecemeal approach We suggest 
that HPT theory-building is best achieved by examining and thoughtfully incorporating the theories of information 
science and organization development. This alignment will foster maximized performance. Our mission is to conduct 
scholarly research in HPT as it plays out in professional practice; a new form of research in the summary is 
proposed, based on “sophisticated practice. ” A HPT job aid is included that provides a synopsis of concepts from IS 
and OD, and their potential applications to HPT. 

Human Performance Technology - HPT - is a rapidly changing field. It has evolved from instructional 
design, so it shares many concepts and applications, but it encompasses a wider and different domain: both training 
and non- training interventions. It is our position that the theories that inform ID are insufficient for HPT; see also 
Davies (1995). Currently, HPT has a limited and shaky theoretical basis; there is no predictive theory, for example. 
HPT continues to change rapidly as its practitioners move from a training perspective/ diagnosis/prescription (based 
in ID theory) to a performance perspective/diagnosis/prescription (based in no particular body of theory, but rather on 
the sophisticated practice and common sense of the practitioners). 

This concept of sophisticated professional practice leading a field is not new to us. Schon has argued for 
some time that in many professional fields the most interesting and potentially useful scholarship is resident in the 
most informed professional practitioners. While accepting this view, we feel that it is not a sufficient state of affairs 
for advancement of the field, and we propose a type of ecumenical theory discussion in the body of this paper and a 
new form of research in the summary. 

A sound theoretical basis for HPT is crucial for several different reasons. First, corporations - the main 
employers of HPT practitioners - are in a massive restructuring process, as a reaction to the changing business 
environment. HPT must change to support business (Professional Concerns Committee Report, 1993; Mirvis, 
1993). To best do so, HPT should have a well-defined orientation and perspective. Second, the HPT field, in order to 
continue educating the next generation of professionals, should have a common ground that unifies all the diverse 
subspecialties - for example, incentive systems to work place design (Gayeski, 1995). Finally, HPT, as a field 
currently rooted more in application than in research, would benefit by being proactive in its research agenda, rather 
than reactive to external forces. This proactivity is best based in grounded theory. A thoughtful and cogent theory- 
building process is preferable to the current piecemeal approach. 

Interdisciplinarity and Convergence 

If HPT needs theory, where should we look for one? A possible solution is to incorporate - or at least 
examine- the theoretical bases of related disciplines. Reality is all of one piece; disciplines continually extend and 
merge into each other (Newell, 1983). The “reality” of organizational performance is examined from the perspectives 
of many different fields, such as finance, marketing, and management. We see the following instance of contrasting 
pertinent theory as one of several possibilities. We hope we will convince our colleagues to consider other such 
efforts to construct a more solid scaffold of relevant theory from adjunct fields. 

Two of these fields, information science (IS), and organizational development (OD) are especially pertinent 
to HPT. First, both IS and OD share much common ground with HPT in terms of contributing fields (Gayeski, 
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1995; Jacobs, 1988; Foshay, 1992; Shrock, 1995, Goffman, 1970). See Figure 1 below. This common ground 
makes IS and OD a natural choice. All three fields have the same overall structured approach in applications: 
diagnosis, intervention, and evaluation. Second, IS and OD complement each other in what they can contribute to 
HPT. IS is focused on information identification, storage, and transmission, and information’s connection to both 
communication and computers. This technological orientation is complemented by OD’s focus on people in 
organizations - how they network and cooperate to get the job done. 

All three fields also have expanding domains of action within the organizational arena. Some aspects of 
current organizational life, which affect or are affected by the disciplines of HPT, OD, and IS are: 

• Compatibility between work objective and work design 

• Information flow between workers and management 

• Appropriate work authority 

• Flexible, multifunctional workers 

• Congruence of reward systems to work design 

• More effective and efficient performance 



Figure 1. Comparison of Contributing Fields: IS , HPT, and OD 
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This figure does NOT show all the contributing fields; it illustrates the diverse range of contributing fields. The 
contributing fields that IS, HPT, and OD have in common are shown in bold. 
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Figure 2. Comparison ofEPSS Functions and 
IS Cognitive Artifacts 
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This figure does NOT show all EPSS functions; only a few are listed to show the similarities. 



This expansion has resulted in a convergence of focus (some would say a collision) and parallel 
development, with both benefits and drawbacks for HPT professionals. A point of convergence between OD and IS is 
in HPTs Electronic Performance Support Systems. Figure 2 compares several EPSS functions and their IS 
counterparts; the terms are different, but the intended functions are becoming virtually identical. 

General Systems Theory and the Social Sciences 

The general systems theory is the father of many contemporary social science theories. It was developed by 
von Bertalanffy in the 1920s, although it did not become popularized until the 1940s (von Betalanffy, 1968). Von 
Bertalanffy originally envisioned it to be applied to the physical sciences, but later it found wide and fruitful 
applications in the social sciences - including the study of organizations (Katz & Kahn, 1966). Von Bertalanffy 
wanted the general systems theory to "unify the sciences and develop the generalized ears ... to overcome the deafness 
of specialization." (Blauberg, Sadovsky, and Yudin, 1977, p 63). General systems theory contributes to HPT, 
information science, and organizational development. 

As most of our readers know, organizations are “systems” composed of interacting elements. These 
elements can be at different levels of complexity, forming a hierarchy of systems. For example, employees in a 
manufacturing department are elements in the manufacturing system; the manufacturing department is an element in 
the organizational system, as are the marketing department and human resources; and the organization itself is an 
element in it’s field of industry. The elements interact with each other and with their environment because the 
system is open to its environment. There are inputs, throughputs, and outputs. Finally, the elements and the 
systems are dynamic, always changing over time. 

The general systems theory found wide acceptance in the social sciences, with each area of study adapting 
the theory to its own unique situation and needs. Although it may not be apparent now, HPT, IS, and OD all have a 
firm basis in the general systems theory (Chems, 1987; Geis, 1986; Jacobs, 1988; Mesarovic, 1983; Rummler & 
Brache, 1988)., This common basis makes theory borrowing tempting and easy - perhaps inevitable. 

General Systems Theory and HPT 

HPT goes beyond training and job aids, in terms of legitimate scope of interventions, to examine the other 
factors affecting individual and organizational performance. These other factors may be anything from the tools and 
incentives (Gilbert, 1978) on the individual level, to the allocation of resources on an organizational level. Rummler 
and Brache (1988) write that the individual is in a “loop of inputs, outputs, consequences, and feedback... ”(p 46) and 
that there are three levels of performance: 

1 . Organizational level 

2. Process level 

3. Individual job level 
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Information Science 

Information science is an interdisciplinary field with diverse contributing fields: general systems theory, 
management science, games theory, cognitive science, computer science, semiotics, cognitive psychology, 
sociology, and communications, to name some, but not all. Some of the same scholars, such as Saracevic and 
Mesarovic, who were instrumental in bringing attention to systems theory were also important contributors to the 
information science field. 

Information science activities are all based in the various phases of the “information life cycle.” If one 
thinks of information as dynamic, changing over time - and also growing old - then the life cycle phases, and their 
pertinence to organizations, become clearer. The information life cycle phases are: 

• Identification of information 

• Capture 

• Codification 

• Storage 

• Maintenance (including additions and changes) 

• Disposal 

IS emerged as a distinct area of research and application during and immediately after World War II (very 
similar to instructional design, isn’t it?). With the advent of modem computer technology, it has become firmly 
based in and concerned with issues of electronic storage and communication. 

IS Trends, Boundaries, and Approach 

IS has traditionally focused on quantity of information processed, not quality of information (Haney, 
1996). In the last ten years, information science scholars and practitioners have shown an increasing concern with 
the larger context in which information technology operates. This is a trend away from data and towards knowledge 
and applications as IS expands its domain. 

Three branches of information science that support organizational performance are: 

1. Information technology (IT): the study of information storage and communication devices, especially computer 
and electronic applications. 

2. Information systems: the study of operations and procedures focused on information. 

3. Information management: the study of information to support decision making. 

The design of card catalogues is a typical example of IS approach. For example, 30 years ago, paper card 
catalogues in libraries were universal; now almost all library catalogues are electronic. The organizing issues about 
the catalogues (How should this information be presented? In what way will patrons try to use this? How should it 
be best accessed? What level of detail should be given on an individual entry? etc.) are typical IS concerns, and have 
not changed, even though the physical artifacts have been transformed. 

Information Science Applications to HPT 

One of the most immediately usable concepts for HPT that IS offers is that of the information life cycle. 
The idea that information has a “shelf life,” and eventually will have to be updated or replaced, provides a new 
perspective on HPT interventions. It stimulates questioning as to where, how, and for how long the intervention 
fits into the organization in a beneficial way. More detailed potential applications of the information life cycle to 
HPT is given in the HPT Performance Tool, which follows this paper. 

Organization Development Components, Trends, and Boundaries 

Because work is done in organizations, it is imperative that we understand how organizations, as social 
systems, impact individual, group, and organizational work performance (Hackman & Oldham, 1980). Cummings 
and Worley (1993) describe organization development as "... a process by which behavioral science knowledge and 
practice are used to help organizations achieve greater effectiveness, including improved quality of life, increased 
productivity, and improved product and service quality. Moreover, OD is oriented to improving the total system ...” 
(p 1). They define OD as “...a system wide application of behavioral science knowledge to the planned development 
and reinforcement of organizational strategies, structure, and processes, for improving an organization’s 
effectiveness.” (p 2). • . 

This concept of OD emphasizes several components that differentiate OD from other approaches to 
organizational change and performance improvement (Cummings and Worley, 1993). For example: 
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• OD applies to the entire system, i.e., company, plant, department or work group, as opposed to only a few 
aspects of the system, such as MIS or individual training. 

• OD is based on behavioral science micro concepts, such as leadership and work design, as well as macro concepts 
such as organization strategy and environmental relations. This is in contrast to approaches which only emphasize 
training, operations management and engineering, which typically neglect the personal and social needs of the 
organization and its members. 

• OD, while concerned with planned change, is not applied in the rigid sense that is normally associated with 
business planning. OD is more of an adaptive process for planning and change rather than a blue print or fixed rules 
for what must be done. 

• OD involves both the creation and subsequent reinforcement of change. It moves from attention to implementing 
change to stabilizing or institutionalizing new behaviors, processes, and activities within organizations. 

• OD encompasses strategy, structure, and process changes, though some approaches focus on one type of change. 
For example, a change process aimed at developing a new strategy might focus on environmental relationships, 
changes in how people are grouped to perform tasks (structure), or problem solving strategies (process). 

• OD is oriented to improve organization effectiveness by first, providing a way for organizations to be able to 
solve their own problems without the help of outside influences and consultants, and secondly, by helping 
organizations realize that to be effective the organization has to have both high quality of work life (QWL) and high 
productivity. 

Organization change is a broad phenomenon involving a diversity of applications and approaches which 
include economic, political, technical, and social perspectives. OD and HPT are not mutually exclusive. We agree 
with Foshay and Moller (1992) that OD is a contributing field to HPT, but we go even further in thinking that OD 
should become more widely incorporated in HPT practices. If properly understood and attended to, OD can provide 
the structure and process requirements to improve HPT interventions at the individual, group, plant, and/or 
organization level. 

OD History 

OD has emerged from five major backgrounds or stems (Cummings and Worley, 1993) beginning in the 
1940s with the growth of the National Training Laboratories (NTL) and the development of training groups, better 
know today as sensitivity groups or T-groups. The second contribution to the field came from the early work in the 
mid-1940s in survey, research, and feedback by Lewin at MIT. Upon Lewin’s death in 1947 his staff moved to the 
University of Michigan and continued working under Rensis Likert at the Institute for Social Research. Another 
contributing factor also took place in the 1940s, which was the classic research performed by Collier, Lewin, and 
Whyte on applying research to managing change. In the 1950s the fourth major contribution came about with the 
focusing on QWL and productivity research by Erik Trist and his colleagues at the Tavistock Institute in London, 
England. Finally, the most recent influence came about in the 1960s through Richard Beckhard’s open systems work 
creating strategic change and organization transformation. 

OD continues to integrate new perspectives, such as strategy, technological change, and total quality 
management. New techniques are continually added to the sophisticated practice of OD because practitioners have 
found that one OD intervention may work in one company or country, but not in another. In our opinion this 
addition of techniques parallels the practice of HPT, and the realization by the fields and the practitioners that every 
intervention, to be most effective, does not necessarily have to be a training intervention, a data base design 
intervention, or a team building intervention. It is this recognition and the ability of the sophisticated practitioner to 
utilize the “correct” intervention to improve effectiveness that will become the cornerstone of maximized 
performance. 

In the field of OD, professional associations have grown from a few people in the initial beginnings of the 
OD Network in 1964 to several thousand members today, and additional organizations, such as ASTD and the 
Academy of Management, have established OD divisions that have thousands of members. The first master’s degree 
program was established in the early 1970’s at Pepperdine University, and the first doctoral program in OD was 
established at Case Western Reserve University (Cummings and Worley, 1993). 

Some noteworthy people that have had an influence on and contributed to the field of OD are: Chris Argyis, 
who developed a learning and action- science approach to OD; Edgar Schein, who tied the key role of culture to 
change management; Edward Lawler HI, who extended rewards and employee involvement to OD; and others, such as 
Peter Block and Thomas Cummings. 



OD efforts have taken place in corporations, schools, governments, and the military. While many 
organizations, both within the United States and internationally, apply OD approaches and techniques (without 
knowing the term exists), few have formal OD programs in place. 

OD Theories and Interventions 

OD is directed at bringing about planned change within organizations. Within this paper we present three 
theories of planned change, one typical OD model of practice, and one intervention system based upon open systems 
theory. 

We understand that most readers will be familiar with these theories. We hope the juxtaposition of the 
theories will provide a fresh perspective, a synergy, that provides new concepts at the theoretical level and new 
possibilities for professional practice. 

The planned changed theories to be discussed are Lewin’s Change Theory model, the Planning Model 
developed by Lippit, Watson, and Westley, and the Action Research Model of Planned Change (Cummings and 
Worley, 1993). The typical practice model presented is a four step model using methods similar to HPT and 
instructional design models. The intervention system based in open system theory that we present is the socio- 
technical system intervention. 

Planned Change Theories 

Lewin’s Change Theory of planned change includes three stages that act to modify those forces keeping a 
system’s behavior stable (Cummings and Worley, 1993). The steps in Lewin’s theory are: 

1. Unfreezing : This step involves obtaining information and activities that help determine the gap between where an 
organization desires to be and where the organization actually is. 

2. Moving: This second step involves shifting the organization or department to a new level through the 
development of new behaviors, and values. 

3. Refreezing: This final step is intended to stabilize the organization at a new state of equilibrium. 

Lewin’s Change Theory could easily be incorporated into HPT as a guideline for successful performance 
interventions and initiatives. Many times a training intervention requires unfreezing, moving and refreezing in order 
to be successful. Too often in the practice of HPT and related fields the practitioner provides the “tools” without 
facilitating the change in organizational, group, or individual behavior that is important to success. Upon 
implementing the intervention the sophisticated practitioner, in our opinion, should make an effort to 
institutionalize the new behaviors that the intervention intended to produce (reffeezing). 

The Planning Model (developed by Lippit, Watson, and Westley) views planned change primarily from the 
perspective of working with the organizational members. The two underlying principles associated with this model 
are: 

1. Information must be freely and openly shared between the organization and the change agent. 

2. This information must result in some type of action. The model has seven steps: scouting, entry, diagnosis, 
planning, action, stabilization and evaluation, and termination. The planned change model is compatible with the 
practice of HPT and provides for a thorough approach to performance. Both HPT and OD practitioners need to have 
removing themselves from the organization (handing off) as an ultimate goal, so that the organization can maximize 
performance on its own. 

The Action Research Model of Planned Change is a cyclical process in which information obtained in the 
initial organizational research is used to foster action, and this action is assessed to provide further information for 
further action, which creates the cycle. Action research traditionally aimed both at helping specific organizations 
implement planned change, and at developing information to apply in other settings. The Action Research Model of 
Planned Change has eight steps: problem identification; consulting with behavioral science expert; data gathering and 
preliminary diagnosis; feedback to key client or group; joint diagnosis of problems; joint action planning; action; 
and data gathering after action. The Action Research approach is worthy of consideration by sophisticated HPT 
practitioners. The joint diagnosing and planning, and identification of new problems that occur as a result of the 
intervention are closely related to the Rapid Collaborative Prototyping Model proposed by Dorsey, Goodrum, and 
Schwen (1995). From an OD perspective it makes sense to include the client in the process of evaluation and 
planning; this is suggested in the guidelines of the Action Research model. 
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A Typical OD Model of Practice 

In the practice of OD, one typical approach includes the use of a four step model, similar to HPT and 
instructional design models. This typical OD approach has four steps: diagnosis, development, implementation, and 
evaluation. This approach has elements of Lewin’s change model, the planned change model, and the action research 
model. While this four step model is easy to remember, it is limited by not focusing on Lewin’s unfreezing and 
refreezing components, and also by not including the joint diagnosis, planning, and cyclical components found in the 
action research model for planned change. 

The Socio-Technical Systems Theory 

OD assesses an organization’s relationship with its external and internal environments. OD also draws from 
the behavioral sciences when it focuses on the individual and group dynamic processes that are essential to 
organizational change and effectiveness. One such intervention system that is based in open system and behavioral 
science theories is the socio-technical system intervention. This theory is based on the observation that an 
organization has both a social aspect composed of people in a social network and a technical aspect composed of 
tools, equipment, and procedures (Schwen, Goodrum, & Dorsey, 1993). “Socio-technical theory is grounded on two 
fundamental premises: (1) that the production of a good or service requires the joint optimization of two independent, 
yet correlative systems - a social and a technical system; (2) that the social-plus-technical system must relate to its 
environment if it is to function and develop.” (Cummings and Srivastva, 1977, p 49). The social component of the 
system includes the social growth and personal interaction needs of the individuals, groups, and organization. 

In order to relate a system to its environment, it is necessary to define the system in a way that 
differentiates it from its environment, and in a way that denotes that the system is relatively open to its 
environment. The environment provides inputs that the systems use as through-puts which return to the 
environment as outputs to be used to provide more inputs to the system. For example, raw materials (inputs) 
become in-process materials (throughputs) which in turn become finished goods (outputs). 

Open systems are hierarchically ordered in that each higher level system is composed of the lower systems. 
From the socio-technical systems perspective, the hierarchy from lower to higher ordered systems is composed of the 
individual, group, and organization (Cummings and Srivastva, 1977). 

Socio-Technical System Design Guidelines 

Using the conceptual underpinnings of socio-technical and open systems discussed above, the following 
guidelines for designing work have been developed by socio-technical practitioners (Cummings and Worley, 1993): 

• Compatibility : designed work should fit the values and objectives underlying the approach. 

• Minimal Critical Specifications: only critical features needed to implement the work should be specified. 

• Variance Control: control technical variance. 

• Boundary Location : organizational boundaries should be recognized to facilitate performance. 

• Information Flow: timely information is essential to control variances and perform tasks. 

• Power and Authority: access to resources and authority to command the resources to perform the work. 

• Multi-functional Workers: personnel should be cross-trained. 

• Support Congruence: systems supporting work design within socio-technical interventions should reinforce the 
nature of the design. 

• Transitional Organizations: a transition plan and system needs to be in place. 

• Incompletion: continual reassessment and change within the system. 

Finally, one socio-technical system approach is associated with self regulating work groups, which are 
groups composed of members performing inter-related tasks. These self-regulating work groups provide the members 
with the skills, autonomy, and information necessary to control their own work with little interference from the 
external environment. “Many organization development practitioners argue that self-regulatory work groups represent 
the work of the 1990s.” (Cummings and Worley, 1993, p 371). 

We think that socio-technical system interventions can help the HPT practitioner focus on macro 
organization perspectives, such as how the organization interacts with the governmental laws and regulations, instead 
of the tendency to focus on the micro organization perspectives, such as training a dock worker in how to use a fork 
lift. In fact, looking at performance from a socio-technical perspective increases the likelihood that the proposed HPT 
intervention will address the performance problems or issues from a total organizational and open system point of 
view. This means that the micro level intervention will become part of the system approach to organizations 
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maximizing effectiveness and therefore, maximizing performance. This systems approach to HPT also assists in the 
design, development, implementation, evaluation, and institutionalization of the performance intervention. 

Summary 

Three different fields, yet so much in common. In our opinion, HPT practice should include the strengths of 
related fields. The demand for information on a just-in-time basis is critical to effective performance. Understanding 
how humans interact with their peers, their organization or group, and their environment is another critical 
performance factor. Well designed training and job aids are critical for performance effectiveness. Most performance 
issues would benefit by being viewed from the “whole” open system approach. While some interventions may 
require training one individual to improve performance, how well this individual fits within the system becomes 
important to the effectiveness of the overall group and perhaps even the organization. 

We believe that Maximized Performance (MP) occurs where HPT, IS, and OD theories converge and 
overlap. See Figure 3 below. It is at this point that optimal balance is achieved because the theories of behavioral 
science, information science, organization development, open systems and performance technology are aligned 
together to foster maximized performance. 

Figure 3 




HPT Research and tKfe Indiana Performance Group 

The Indiana Performance Group (IPG) was recently formed. To address the dilemma of the best scholarship 
in our field being rooted in advanced professional practice, our mission is to conduct scholarly research in HPT as it 
plays out in professional practice. We look forward to working with internal and external consultants; using both 
primary and secondary research methods; and emphasizing measurement. We will attempt to “capture” sophisticated 
practice by reviewing case studies, original documents, and interventions. We believe that, if we can assure 
proprietary rights, the careful and disciplined showing of advanced professional practice will move theory in new and 
unexpected directions. The IPG will use the information collected to inductively discover patterns of sophisticated 
practice. We ask that any readers willing to share their insights on theory and/or practice contact the Indiana 
Performance Group 

c/o Instructional Systems Technology Department 
School of Education 
Indiana University 
Bloomington IN 47405 

References 

Blauberg, I. V., Sadovsky, V. N., & Yudin, E. G. (1977). Systems Theory: Philosophical and 
Methodological Problems . Moscow: Progress Publishers. 

Chems, A. (1987). Principles of Sociotechnical Design Revisited. Human Relations. 40 . 153-162. 
Cummings, T., & Srivastva, S. (1977). Management of Work: A Socio-Technical Systems Approach . Kent, OH: 
Comparative Administration Research Institute of Kent State University. 




62 



69 



Cummings, T. G., & Worley, C. G. (1993). Organization Development and Change . (5th ed.). New York: 



West. 

Dorsey, L. T., Goodrum, D. A., & Schwen, T. M. (1995). Rapid Collaborative Prototyping as an 
Instructional Development Paradigm. (Unpublished Manuscript ). Bloomington, IN: Indiana University. 

Davies, I. K. (1995). Re-Inventing ISD. In B. Seels (Ed.), Instructional Design Fundamentals: A 
Reconsideration (pp. 31-44). Englewood Cliffs, N.J.: Educational Technology Publications. 

Foshay, W. R., & Moller, L. (1992). Advancing the Field Through Research. In H. D. Stolovich & E. J. 
Keeps (Eds.), Handbook of Human Performance Technology: A Comprehensive Guide for Analyzing and Solving 
Performance Problems in Organizations, (pp. 701-714). San Francisco: Jossey-Bass. 

Gayeski, D. M. (1995). Changing Roles and Professional Challenges for Human Performance Technology. 
Perfo man c e Im pr ovement Q uarterly* 8 (2), 6-16. 

Geis, G. L. (1986). Human Performance Technology: An Overview. In M. Smith (Ed.), Introduction to 
Performance Technology (pp. 1-20). Washington, D. C.: National Society for Performance and Instruction. 

Gilbert, T. (1978). Human Competence: Engineering Worthy Performance . NY: McGraw Hill. 

Goffman, W. (1970). Information Science: Discipline or Disappearance. Aslib Proceedings. 22 ( 12 ). 589- 

596. 

Hackman, J. R., & Oldham, G. R. (1980). Work Redesign . Reading, MA: Addison-Wesley. 

Haney, D. S. (1996). Corporate JCnowledge Asset ManagemeoUand Multidisciplinaritv . (Unpublished 
Manuscript ). Indiana University: Bloomington IN. 

Jacobs, R. L. (1988). A Proposed Domain of Human Performance Technology: Implications for Theory and 
Practice. Performance Improvement Quarterly, 1 (2), 2-12. 

Katz, D., & Kahn, R. (1966). The Social Psychology of Organizations . NY: John Wiley & Sons. 
Mesarovic, M. D. (1983). Mathematical Systems Theory and Information Sciences. In F. Machlup, and 
Mansfield, U. (Ed.), The Study of Information (pp. 567-571). NY: Wiley-Interscience. 

Mirvis, P. H. (1993). A Competitive Workforce: The Issues and the Study. In P. H. Mirvis (Ed.), Building 
the Competitive Workforce: Investing in Human Capital for Corporate Success (pp. 1-30). NY: John Wiley & 
Sons. 

Newell, A. (1983). Reflections on the Structure of an Interdiscipline. In F. Machlup, and Mansfield, U. 
(Ed.), Th e S tudy o f I n fo rmati on (pp. 99-109). NY: Wiley-Interscience. 

Professional Concerns Committee Report. (1993). ASTD News & Notes. 6 171. 1. 

Rummler, G. A., & Brache, A. P. (1988). The Systems View of Human Performance. Training. 25 (9). 45- 
53. 

Schwen, T. M., Goodrum, D. A., & Dorsey, L. T. (1993). On the Design of an Enriched Learning and 
Information Environment (ELIE). Educational Technology. 33 01). 5-9._ 

Shrock, S. A. (1995). A Brief History of Instructional Development. In G. Anglin (Ed.), Instructional 
Technology; Past. Present, and Future ( 2nd ed., pp. 1 1-18). Englewood: Libraries Unlimited. 

von Betalanffy, L. (1968). General System Theory; Foundations, Development, Applications . NY: 

Braziller. 

Biographical Information 

Debra Haney is the Manager of Haney Consulting, and consults in the areas of information technology 
and performance technology, designing and developing performance improvements. She may be reached at Haney 
Consulting, P. O. Box 674, Bloomington, IN 47402, (812) 334-0185, or e-mail dhaney@indiana.edu. 

David R. Dent is the President of Locus Technologies, inc., a consulting company in strategic planning, 
change management, and organizational effectiveness. He may be reached at 1415 E. 3rd St. #130, Bloomington IN 
47401, (812) 857-6936, or e-mail ddent@indiana.edu. 

Thomas Schwen is the Chairman of the 1ST Department at Indiana University. Tom has over twenty- 
five years experience consulting in strategic planning of human performance organizations. He may be reached at the 
Instructional Systems Technology Department, School of Education, Indiana University, Bloomington IN 47405, 
(812) 856-8450, or e-mail schwen@indiana.edu. 



ERJC 



63 



70 



HPT Performance Tool 



Note: this performance tool for human performance technology practitioners is organized as follows: first, it 
presents the main components of theories and models from information science and organizational development. 
Next, it presents the potential applications to HPT of those theories and models. Last, it gives space for the 
reader to write in personal applications. We hope that this “job aid” will benefit our HPT colleagues by 
suggesting some new ideas and approaches to the situations they encounter at work. 



. ’ V</" . Idea Synopsis 


Potential Applications to HPT 


. v V Personal Applications , 
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The Information Life Cycle 






1. Identification of information important 


• Front End Analysis: the preliminary 




enough to be saved. 


investigation. Addressing the issues of what 
the situation is, what type of intervention 




2. Capture of information. 


will be appropriate, who are the subject 
matter experts, and what is the best way to 
obtain the resources necessary to the 
project. 




3. Codification of information in a form 


• Intervention design. Addressing the 




that will be secure and appropriately 


issues of intervention format and 




accessible. 


presentation, including incorporation of the 
organization’s culture. 




4. Storage of that information. 


• The intervention implementation. 




5. Maintenance of the information. This 


• Organizational responsibility for the 




includes additions, changes, and 


project after hand-off. 




deletions. 


Addresses questions such as: Who shall be 
responsible for maintenance of the HPT 
intervention? Is the person committed and 






empowered (in other words, does the 
person have both the responsibility and the 
authority)? 




6. Disposal 


• Disposal decisions. 

Addressing the issues of disposal 
responsibility and criteria. 




OD Interventions 






The following are client groupings that may 


The following are examples of OD 




require an OD intervention to improve 


interventions for each client group that 




effectiveness: 


apply to improving performance: 




1. Individuals 


• Role expectation and clarification 

• T-groups 




2. Within Teams/Groups 


• Teambuilding 

• Role expectation and clarification 




3. Between Groups 


• Survey feedback 

• Interviews or questionnaires 




4. Total Organization 


• Strategic planning activities 

• Team building at all levels 




s* ' 7 ■ 

Idea Synopsis 


Potential Applications to HPT 


.• Personal Applications ;• 7 . « « 
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Le win’s Change Theory 

This theory provides a model for 
unfreezing, moving, and refreezing the 
behavior of the individual, group, or 
organization in order to facilitate 
implementing and sustaining change within 
the organization. 


1 . Identifying and reducing resistance 

2. Framework for understanding 

organizational change 

3. Improving the effectiveness of the 

individual 

4. Improving team effectiveness 

5. Developing safety and trust within 

organizations and between groups 

6. Developing and implementing new 

reward systems 




Action Research Model of Planned 
Change 

This model of planned change has three 
distinguishing characteristics: 

• Consultation with a behavioral scientist 

• Joint diagnosis and action planning 
between the consultant and the client 

• Cyclical in nature 


1. Restructuring an organization due to 

new or changing governmental 
regulations 

2. Reengineering at the organization, 

division, SBU, plant, and/or group level 

3. Organizational mergers 

4. Integrating training, HR, and strategic 

planning into one unit 

5. Start-up a new division of an 

organization 

6. Developing and implementing new 

reward systems 

7. Determining production training 

requirements within a plant 

8. Integrating with strategic management 




Socio- Technical Systems Interventions 

Two important characteristics of socio- 

technical interventions are: 

• An organization has both a social and 
technical system in place that need to be 
attended to for maximized performance 

• Organizations interact with their internal 
and external environments 


1 . Identifying the need for teams (and the 
type of team, if any) for new 
manufacturing techniques 

2. Developing training requirements for 

people whom are not team oriented 

3. Improving individual effectiveness 

4. Improving team effectiveness 

5. Developing training and strategies for a 

new product introduction 

6. Developing and implementing new 

reward systems 

7. Strategic management 
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Employing Cognitive Tools within Interactive Multimedia 

Applications 



John Hedberg 
Barry Harper 
Robert Wright 
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Abstract 

This paper describes research into the use of cognitive tools in the classroom using Exploring the Nardoo , 
an information landscape designed to support student investigation. Simulations and support tools allow multimedia 
reporting and are supported by several metacognitive tools for the writing process, which provide scaffolding 
templates to support the learners. 

Introduction 

The introduction of information and telecommunication technology, and specifically computers, into the 
educative process has often been heralded as the new panacea for education. The technology offers teachers the 
opportunity to individualize instruction, place children in open ended student-centred investigations, and to shift 
from their traditional instructor role to mentor and co-learner. The panacea however, like past revolutions in 
education, will go the way of previous technologies unless there are changes to the cognitive tools provided within 
or supporting complex applications. Alfred Bork (1995) has argued in his critical review of the failure of computers 
in schools and universities that the effective use of new instructional paradigms requires: 

* a shift in teachers’ pedagogical approaches 

* software that supports the modes of instruction that cognitive scientists are telling us are appropriate 

One of the most extensive longitudinal studies on the effect of the introduction of information technology 
in classrooms has been carried out under the Apple Classroom of Tomorrow (ACOT) program (Dwyer, 1995). The 
ten year research and development collaborative program focused on how teaching and learning changed when 
teachers and students have routine access to technologies. One of the key elements of this program was a belief that 
technology should be used as a tool for learning and a medium for thinking, collaborating, and communicating. The 
outcomes of the program have been summarized by Dwyer (1995) 

...technology plays a catalytic role in opening the minds of teachers to new ideas about children, learning, and 
their own role in the education process. 

Dwyer also claims that without this form of reflection and subsequent changes in professional practice, the promise 
of technology will never be realized. This premise supports the criticisms of Bork (1995) and Schank and Cleary 
(1995) of our current modes of education. 

Changing instructional strategies with Interactive Multimedia 

Instructors have traditionally presented a linear narrative sequence which revealed the underlying structure of 
their ideas, based largely upon their understanding of the concepts, and their perception of the learning environments 
they have generated. However, learning environments in classrooms are affected by the behavior of teachers which 
are the outcomes of the beliefs and goals a teacher brings to the classroom (McRobbie and Tobin, 1995). So too do 
the beliefs of students result in a diverse set of perceptions of the classroom learning environment (Roth & 
Roychoudhury, 1992). Ritchie et al (1995) have found, in exploring the boundaries of learning environments in a 
science classroom, that individuals and groups experienced different learning environments and that the actions of 
students and their teacher could be explained in terms of their beliefs, roles, goals and behaviors in various activity 
settings. 
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Current interactive multimedia technologies can represent ideas in almost any mediated form, and provided 
we can generate a comprehensible metaphor for organizing our functional options and the underlying knowledge 
structures, the student can roam through the resources, creating their own meanings and understandings of the 
phenomena they encounter, ie creating their own form of the learning environment rather than one generated by their 
teacher or by the package designer. With graphical and visual display coupled with large databases of resources, it is 
possible to explore an information space in whatever sequence appeals as appropriate to the user or to the specific 
task. When raising the idea Florin (1990, p30) saw information landscapes, ...as virtual towns, or intellectual 
amusement parks. The analogy is quite intriguing and helps us to visualize many abstract concepts within a single 
metaphor. 

This form of representation of information supports students 1 learning processes advocated by researchers 
like Schank and Cleary (1995) who have argued strongly for the use of such technology to support students in 
following their own interests or questions. This rich context has the potential to allow the novice to work with 
authentic problems and practice. 

However, within this context, designers of multimedia learning environments have tended to be narrow in 
their view of how users will interact with the rich array of multimedia resources once a challenge, in the form of a 
problem to solve, has been posed. Instructional designers have not often taken full advantage of the technology that 
is being used to present these powerful ideas. Once the material has been presented to user(s) and they have interacted 
in the ways envisaged by the instructional designer (and often in new ways not considered by the designer) the user is 
left to ponder and present their conclusions using more routine presentation technologies, such as pen and paper. 
Increasingly, users have access to the same multimedia technology but have lacked access to the rich digital media 
resources embedded in the learning environment. 

As new theoretical views of learning have developed, it has been recognized that learners act as active 
constructors of knowledge (Duffy and Jonassen, 1992). Within this constructivist framework, which is concerned 
with the process of how we construct meanings of our world as well as with the results of the constructive process, 
cognitive tools can help learners organize, restructure and represent what they know. Jonassen and Reeves (1996) 
have summarized the foundations of cognitive tools research and have identified the following key principles in the 
context of multimedia design: 

• Cognitive tools will have their greatest effectiveness when they are applied to constructivist learning 
environments. 

• Cognitive tools empower learners to design their own representations of knowledge rather than absorbing 
knowledge representations preconceived by others. 

• Ideally, tasks or problems for the application of cognitive tools should be situated in realistic contexts with 
results that are personally meaningful for learners, (p. 698) 

Additionally cognitive tools to support the user have been shown by Jonassen (1995) and others to enhance 
the learning process and to support the users’ investigations. If students are to truly create their own meanings and 
understandings of the phenomena they encounter, designers need to not only incorporate user tools which will enable 
them to present their findings using the full array of resources contained in the packages, but also support their 
investigations with powerful cognitive tools. 

The lack of powerful learning environments embedded in much of the interactive multimedia products 
currently available cannot be entirely attributed to the lack of understanding of the results of cognitive science 
research by developers. The challenge for researchers is to not only conceptualize powerful learning environments, 
but also demonstrate that process so that developers of educational software produce products that support learning in 
the most effective ways rather than predetermine the learners’ needs, interests and preferred learning styles. This use 
of information technology offers the opportunity to shift the learning to student driven and directed learning. 
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Supporting learning with Cognitive Tools 

Cognitive scientists are attempting to narrow the gap between the learning environments portrayed in many 
commercial interactive multimedia packages and learning environments that will truly enhance learning. Schank and 
Cleary (1995) have described a set of innovative learning architectures based on their conceptualization of realistic 
learning situations. These powerful packages illustrate the lessons of cognitive science research through: 

• Simulation-Based Learning by Doing 

• Incidental Learning 

• Learning by Reflection 

• Case-Based Teaching and 

• Learning by Exploring 

Each of these teaching architectures have been illustrated with specific packages such as Dustin , a simulator 
designed to help a student learn a foreign language, Road Trip: The Geography Un-Lesson, Sounding Board , a 
package to bounce ideas off, Creanimate Project, a case-based package for designing/creating animals and ASK 
System, a form of hypermedia based on the metaphor of having a conversation with an expert. The software 
technologies discussed by Schank and Cleary (1995) create powerful example implementations of cognitive tools 
where different cognitive learning strategies are built into software and the learner is encouraged to explore their ideas 
and solutions with differing degrees of support and advice. 

The innovative use of cognitive tools in interactive multimedia learning environments has also been 
reported by Lajoie and Greer (1995). The package Bio- World (Lajoie, 1993) is an interactive learning environment 
designed to support the acquisition of scientific reasoning skills in high school students and integrates a variety of 
cognitive tools to assist in scaffolding scientific reasoning activity. Users of this package are engaged in explicitly 
justifying hypotheses with evidence; organizing, categorizing, and rating evidence; and constructing a final summary 
argument on the topic of bacterial and viral infections. A proposed development for this package incorporates an 
authoring mode for students to generate new scenarios for their peers to investigate, this in turn, will support the 
powerful augmentation framework design of the package. 

Exploring the Nardoo with student driven investigations 

With an understanding of the shortcomings of much of the commercially generated learning packages, we 
sought to combine the ideas of situated learning and problem based learning from rich information landscapes to 
form the basis for effective design. Within this context we also sought to incorporate a range of cognitive tools 
within the landscape which would contribute to supporting the learner. The detail of this design process and the 
theoretical position has been reported in Hedberg et. al. (1994). The resulting package, Exploring the Nardoo , 
provided a rich information landscape of resources based on ecology. The information landscape uses a geographic 
metaphor based upon a Water Research Center and a navigable river environment. It incorporates problems that 
challenge students to become active participants in the learning process and simulators that allow the user to ask 
questions and investigate possible answers to those questions. By providing a metaphor relating to the real world, 
students are encouraged to apply scientific concepts and techniques in new and relevant situations in this ecology- 
based application, throughout the problem-solving process. In so doing, the learner is likely to become more 
interested in developing questions, ideas and hypotheses about the learning experiences encountered. As an alternative 
teaching/leaming strategy in the development of inquiry and problem solving techniques, this package incorporates 
high quality visual materials in the form of graphics, sound, text and motion video together with scientific 
measuring tools to aid in the construction of understanding. Exploring the Nardoo provides the student with a 
flexible set of tools made available through a personal digital assistant (PDA), Figure 1, to assist in the 
investigation process. 






Figure 1. The Personal Digital Assistant 
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Figure 2. Water Management Simulator Notebook 



The process of using source material within the package in support of an investigation has been enhanced to allow 
the student to: 



• Decide precisely on the quantity and selection of text to be copied into their notes. This is either through 
making a selection and then ’grabbing’ it into the PDA or by using a ’drag and drop’ technique where the target 
text is selected or highlighted and ’dragged’ into the notes module of the PDA . 

• Use marker buttons as pointers to video, audio or picture information which can be displayed within the 
PDA’s viewer along with any linked information. For example, by copying a picture of a wombat into their 
notes, the student is able to move through the information landscape in the package and very quickly view the 
picture of the wombat as well as its associated text in the PDA’s viewer merely by clicking on the marker 
button within their notes. User defined portions of the reference text material displayed within the viewer 
may be selected and copied into the notes also. 

• Manipulate marker buttons and text within the notes areas, via ‘drag and drop’, to facilitate the re-ordering of 
ideas in the process of building an investigative response in the form of a report, explanation, procedure etc. 

• Use text style tools, within editable text notes, providing the opportunity to use font color, style and size as 
organizing criteria within the notes. For example a student may recognize that a certain combination of text 
attributes is representative of newspaper clippings or they may choose to color information they write or 
gather from a particular perspective in a special color. 

The joint combination of note book and viewer equips students to view and then critically evaluate or 
compare different representations of the same information concept. By collecting different media representations of 
the same topic and ‘flipping’ between these representations at their discretion, the student has the opportunity to 
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establish cognitive links between different media forms which complement each other and support a central theme or 
information focus. 

The package also provides the ability to record thoughts and impressions ‘on-the-fly’ whilst examining 
media stories. This provides the potential for students to reorganize or revise their thoughts to better ‘make sense’ of 
what they see and hear. Students are able to document their emerging ideas in support of an investigation or 
problem solving exercise whilst viewing different media. This provides support in the formulation of new schemata 
in the process of accommodating the new information. 

By way of illustration, whilst viewing a video segment related to the issue of water conservation, students 
may have their attention drawn to a specific aspect such as better ways to manage water use within the household. 
The Exploring the Nardoo PDA provides the means by which they are able to search out other related information 
(text, graphics, radio/television reports) and store it within their notes ‘alongside’ previously captured information. 
The different perspectives represented within the source media items can be viewed, edited and brought into context 
within the student’s framework of notes they are making. 

Successful problem solving activities are reliant on numerous individual, social and environmental factors. 
From a technique perspective. Exploring the Nardoo endeavors to assist students by providing some structures, or 
templates, upon which they can build their note taking or response writing activities. These are in the form of 
writing genre templates. Students may access the book containing these templates (as well as other organizational 
help on note taking, presenting and filing) from within the Water Research Center - a metaphor within the 
information landscape of the package. Genre descriptions can be viewed and a genre template can be copied into the 
notes and used as a scaffold upon which to build or fill-in relevant information found whilst exploring the package. 

To facilitate the re-ordering or re-prioritizing of information Exploring the Nardoo provides a separate, 
expanded form of the notebook. This device has been termed a ‘text tablet’. It provides the editing facilities offered 
by the PDA as well as other features to assist with the restructuring of notes into a form more suited to small group 
presentation or a particular genre style. The text tablet provides a larger expanse of editable screen/document space 
into which student notes may be copied to/from the PDA notes module. 

A writing genre template (discussed in the following section) can also be loaded directly into the text tablet 
into which portions of the student’s notes may be copied or dragged. Notes from prior sessions can be loaded into 
the text tablet and used in support of current investigations. Being able to store and report thoughts and impressions 
derived from media experiences by using the media itself (actual video/audio and pictures, not just text 
representations of the media) provides a more powerful means of ‘reformulating’ (Schroeder & Kenny, 1994, p 965) 
ideas. 

The multimedia collection, editing and presentation facilities offered within Exploring the Nardoo are 
extensive and present a great potential for students to become manipulators of multiple media. New avenues are 
opened for expression for those who choose to use the facilities within the package. We are aware however, as 
Schroeder and Kenny (1994) point out “learner’s not accustomed to this technique and multimedia facilities will 
require instruction in its use” before they become proficient with the technique but once accustomed to it the student 
has a powerful process at their disposal to gather, organize and illustrate their ideas. Support for teachers and students 
in the use of these features has been modeled using walk-through movies made available through the help system 
and also detailed in support notes available in reference books within the package. 

Specialized support tools for exploration 

Exploring the Nardoo contains two specialized tools aimed at providing support at a deeper level for the 
exploration process, namely, three interactive simulations and a presentation guide which, together with support for 
note taking methods and suggestions on filing of materials, contains a series of genre templates. 

The three simulations available in Exploring the Nardoo are, an algal bloom simulation, a whole catchment 
level, water demand management simulation and a personal, home based water use simulator. Each of these 
simulations is a powerful exploratory tool, which provides support for the solution to one of the embedded problems 
by mimicking a Teal world process’ which forms an integral part of one of the problems encountered in the Nardoo 
River Catchment. They greatly enrich the ‘quality’ of the problem solving process for students by providing the 
student with unhindered access to act and become immersed in a ‘real’ situated process, manipulating the various 
causal parameters and testing hypotheses without a ‘real’ consequence or risk and in a time frame which is 
convenient to them. (Figure 2) 

They promote the adoption by the student of the active learner mode and in so doing support the active 
construction of knowledge by the student during the process of solving a problem. The simulations complement the 
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problems embedded in a complex information landscape by providing links with the ‘real world’ experience and by 
creating an environment in which the user may practise skills. 

More able students are provided with the facility to solve problems at a deeper level through the testing of 
their own “what if’ scenarios. This can, during the course of solving problems, facilitate more detailed exploration 
and learning by: 

• allowing the user to take readings at a site which they use as inputs for the simulation studying the changes 
as the simulation runs, 

• allowing the monitoring of all parameters while the simulation is running, with the aim of exploring the 
relationships between them (Corderoy, Harper & Hedberg, 1993, p 126) 

A necessary skill in problem solving is the ability to collect and manipulate and analyze many different 
forms of data and then present them in a meaningful and useful way to any of the many different discourse 
communities. The genre template tool in the presentation guide provides a suitable mechanism for the support and 
development of this skill. 

The student support for the collection of this data in the form of note taking and the resulting solutions to 
problems needs to include modeling of the various styles of discourse used in different communities. 

The use of the notebook metaphor can be carried through to the creation of multimedia presentations. The notebook 
can be used as an organizer for drafting the presentation and should make the transition to a finished multimedia 
product seamless. Templates such as the genre templates within a notebook can be used as advanced organizers for 
the learner, especially those with little prior knowledge. The templates also provide a framework for gathering 
information and stimulating recall of prior knowledge. 

By making such templates available and encouraging their use, we are assisting students through a modeled 
form of outlining. Identifying concepts within their notes that bear some relationship to part of a template structure 
requires high order thinking skills which “a) causes focusing on important points, b) helps students gain familiarity 
with text structure, c) aids retention, d) generates useful alternative texts to supplement materials read, and e) causes 
active participation in learning (Bianco and McCormick, 1989 in Schroeder & Kenny, 1994, p966) The value of 
this modeling process is not faculty, learning style, level of school, or type of writing dependent. 

The development by an individual of genre-specific schemata can have a number of generative and 
empowering consequences, the most significant of which is the development of the ability to communicate more 
effectively with a wider spectrum of the community by producing better responses to problems. 

Researching the cognitive tool use by students 

To investigate the use of these tools by students a series of data gathering methods were employed. Two 
groups of students (n= 8 and 11) from a local high school were organized to attend a work session where they 
followed the procedure below. 

1 . Students were given a short introduction and orientation to the staff present and the facilities they were going 
to use. 

2. A demonstration of the package was provided using a large projected display on which all visual and auditory 
clues could be seen and heard clearly. At this development stage, it was not possible to have an in-built 
“Help” facility within the package, the demonstration covered a basic walk-through of the package 
showing... 

• a representative sample of accessing information in different areas leaving sufficient room for student 
exploration and experimentation, (eg in the Water Research Center students were shown how to activate a 
‘hot-spot’ and ‘zoom-in’ to view a Plant and Animal Book; in a region of the river environment students 
were shown how to activate the tools module to measure embedded environmental data and also how to 
gather an embedded media article.) 

• ways of navigating around the package 

• general operation of the Personal Digital Assistant (PDA). 

3. The students were introduced to the survey form which they were asked to complete during and after the use 
of the software. All questions were answered. The survey was aimed at the functionality of the different 
program elements with specific attention being paid to the Personal Digital Assistant. 

• Questions were brief with ample room and encouragement for individual comments or general reactions 
to the program. 
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• Direction was given within the survey to spend about 20 mins using the facilities within the Water 
Research Centre and the remainder (approximately 25-30 mins) investigating a problem. 

• Students were encouraged to write any other comments they may have had about their experience that fell 
outside the questions on the form. 

4. The students were then asked to work in pairs and in some cases a group of three (this sort of collaboration 
is encouraged in working with this program). As a team students worked their way through the 
worksheet/survey at their own pace. 

5. During the activity the teacher, the researcher and other staff moved about providing assistance if requested 
and making observations as to how the package was being used. 

6. At the conclusion of the allocated time, the students were once more, either in their small work group or 
together as a unit, able to express impressions, concerns, praise etc arising from the session. 

7. The class teacher was given opportunity to review the written responses of the students and to give some 
context to them relative to the ‘perceived’ way in which he thought the students may or may not have 
reacted to the activity. 

Results 

One of the most obvious results that had a direct influence on the direction and scope of this investigation, 
was the need for students to become familiar with the package and how it functioned before they are able to attempt 
to develop a response to an investigation. In the time allocated students did not seem comfortable enough with the 
facilities or general concept of the package to become seriously involved in using the multimedia editing tools to 
edit their work. Most of their time was spent exploring and gathering resources — this process was lengthened as 
they were simultaneously completing the survey form. The students were able to focus more on investigating an 
issue when they were familiar with the package and more resources were embedded. This was an indication of the 
high level of intuitive functionality that the package possessed. This investigation is continuing. 

This research study also identified the need for more specific details about how each program element 
functions and for clearer statements of what problem solving strategy should be selected and how it might have been 
implemented in response to an embedded investigation problem. These issues have largely been addressed through 
the visual modelling of processes and tool functions by ‘ walk-through ’ movies in the ‘Help’ system of the package 

From the package description above and the data gathering activities, the level of successful problem 
solving relied heavily the functionality of the visual images and pseudo-measuring devices. It was also apparent that 
until the package was fully resourced in terms of ‘Help’ and data gathering/organising features, the use of the tools 
for report generation or idea re-organisation required substantial support external to the software. It was apparent that 
students would probably need to seek assistance from their teacher when using a program of such density of 
information provision. However, some problem solving help would overcome widely varying levels of teacher 
expertise. Some questions that will be investigated in the future include: 

• Now the program is fully resourced and functioning intuitively, the level of acceptability of the interface and 
degree of difficulty students encounter when using the editing and presentation tools will be assessed. 

• Video and audio recording of test subjects with subsequent transcription and analysis of the sessions will 
enable a more detailed analysis. 

• Students will be asked to draw a concept map (bubbles, boxes, labels and arrows) to represent their perception 
of what the package contains and any interrelationships that exist at the end of their session of working with 
the materials. 

Some ideas have been gathered and already implemented as a result of the data gathered. For example, a 
more easily identified button on the Personal Digital Assistant to access the note-editing tools, and clearer direction 
when developing necessary instructions for using the simulator and text tablets. Favourable comments have been 
received about the functionality of most of the resources within the Package. Observations revealed that not all 
students realised that the clipboards could be used to access video and radio material, and could be viewed in the 
Personal Digital Assistant viewer by clicking on the highlighted line of text which described a particular media item. 
Some students just read the text and turned the pages. This indicates the need for further instruction on the 
clipboard. The need for a clear demonstration showing investigative activity and appropriate problem solving within 
the package is still required. 
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Future Cognitive Tool Development 

The outcomes of the research so far suggest that the cognitive tools designed to support Exploring the 
Nardoo, have the potential to support learners in the quest for meaning and understanding. However, it is anticipated 
that a range of further tools could extend the support for users. For example, the work by Ferry (1996) has shown 
that cognitive tools with a semantic emphasis can be used effectively to represent the developing knowledge of 
learners. His concept mapping tools have shown the potential to support learners in structuring and representing 
their ideas. Exploring the Nardoo currently contains cognitive tools that allow users to represent their ideas in 
multimedia format, but only in a linear form. A concept mapping tool which enabled non-linear relationships to be 
represented would help students to constuct a more complete and visual representation of their ideas. 

Other tools that will now be investigated include: 

• Resource collection tools that enable the collection of any visual representation format: verbal, data, graphics 
or temporal (through Quicktime movies) 

• Conversion tools that enable changing of information from one representation form into another, either by 
making notes or the permanent link between a visual representation and its textual description 

• Oganising tools to enable different representations of the problem solution 

• Scaffolding tools that anable students to organise their concepts into a logical structure which might depend 
upon the form into which knowledge is to be transformed. 

Conclusion 

The development of innovative interactive multimedia packages that support student driven exploration and 
investigation has not been a priority for the multimedia industry. Design models and examples of such products are 
now being presented by researchers to not only support the research endeavour in this field, but also to support more 
commercial design of educational products. Exploring the Nardoo is an example of a product which provides a range 
of cognitive tools in an information landscape to support student investigation. Simulations and support tools 
which allow multimedia reporting are embedded in the package and are supported by several metacognitive tools for 
the writing process. These tools not only include details about genre but also scaffolding templates to support the 
learners. The extent to which problem solving and student centred learning goals are achieved will be investigated 
and reported upon as the use of the product within schools becomes more widespread. 
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Whether we like it or not we can never sever our links with the past complete with all its errors. -Ludwik Fleck, 
1979, p. 9. 

Abstract 

This paper reports on some of the important considerations that could characterize historical study in the 
field of educational technology. It will begin with a brief description of intellectual history. This description will 
include a deeper discussion of the domains of intellectual history. This in turn will be followed by a discussion of 
the relationship of theory to method in historical studies. The paper will conclude with hints about using conflict as 
a way to study the history of the academic field of educational technology. 

Introduction 

Many professionals seem to be interested in the history of the field in which they practice. Some are 
interested in knowing their intellectual heritage, others would merely like to have a sense of their professional roots. 
But recent studies (Young and Januszewski, 1990; Caffarella, 1992; Januszewski, 1994) suggest that there have not 
been many historical studies conducted in the field of educational communications and technology by members of the 
field. One can get a feel for the history of the field of educational technology by reading Paul Saettler's The 
evolution of American educational technology (1994). But this encyclopedic approach to the history of the field does 
not attempt to analyze the intellectual heritage of educational technology. This is the domain of intellectual history. 

We usually think of historical studies as focused on the political, military, and economic aspects and 
accomplishments of a society. In the earlier part of this century, scholars began a movement to study the 'thoughts' 
of individuals with the hope of developing a broader characterization of the people that live in particular culture at a 
given time period (Veysey, 1977). This movement became known as "intellectual" history. The idea of intellectual 
history has spread to a number of arenas including the history of education, and more recently, the history of 
educational technology (Januszewski, 1994). 

Studying the intellectual history of the field of educational technology should help to answer the questions 
like 'why do educational technologists think the way they do about educational technology?'; 'how has that way of 
thinking changed with time?'; and why is the field of educational technology the way it is?'. Perhaps these questions 
appear to invite mere speculation but as historian George Boas stated, "history tells us among other things how we 
got to think the way we do-and if that is not of importance one wonders what is" (Boas, 1969, p. 3). 

What is intellectual history? 

Briefly put, intellectual history is the history of thought or some specific aspect of thought. Intellectual 
history has been divided into three primary areas of study; biography, the history of ideas, and the history of culture 
(Higham, 1977). Members of the field of educational technology would be most familiar with biography (Human 
Performance Quarterly and Media and Technology Yearbook). Biographical studies focus on the thoughts, work, and 
lives of single individuals. 

The history of ideas is often a cross disciplinary or perhaps multi-disciplinary endeavor. Here scholars break 
out of traditional disciplinary boundaries seeking important or "great works". These great works are coherent 
expressions of thought about some topic in question. These illustrate the efforts of certain exceptional individuals. 
Exceptional because, by definition, not everyone can produce a great or exceptional work. The study of the great 
works of a field requires a thorough investigation because one must analyze the content of the work, the context in 
which the work was written, and the intent with which it was produced. 

The history of culture emphasizes the "collective mentalities or thought collectives" that exist within given 
societies at different points in time (Wood, 1977). Cultures are part of a larger society. Studies in the history of 
culture focus more on the development of the consciousness of a group and the many topics or influences that help 
to shape the thought processes of a group. The notion of a collective mentality can be used to describe certain 
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localized racial, ethnic, and socio-economic groups of individuals, also intellectual fields and disciplines, etc., such as 
educational technology. 

The difference between the history of ideas and the history of culture is a matter of degree. Studies in the 
history of ideas and the history of culture often include both, the biographical aspects of individuals who were 
important to the formulation or a change of a certain idea or conception or a groups way of thinking about an idea. 

The essential difference between these last two approaches has to do with the level of conscious thought 
that a historian chooses to highlight (Veysey, 1977). Although these two approaches to intellectual history have 
different purposes, they complement and support each other (Higham, 1977). It would seem that the history of ideas 
plane is easier to study than the history of culture. This is because views are expressed there, or the material being 
studied, was usually intended to be made more clear and precise. The history of culture is, at least on face, a bit 
messier. But a good study in intellectual history will consider aspects involved in both of these approaches. There 
is much to be learned from the study of how consciously articulated ideas become a part of a "thought collective" in 
a given field of study and how a thought collective influences the work of an individual (Fleck, 1979; Kuhn, 1970). 

Theory and method in historical study 

In a broad and varied field such as educational technology where each person formulates her or his of view of 
what the field is, each person could have a correspondingly different view of what the field's history is. The 
existence of a plurality of histories is not itself a problem. The problem arises when large numbers of members of 
the field believe that only certain factual content is the limit of the study of the history of the field. Recognizing the 
fact that there are a number of approaches to studying history brings us to a discussion of theory and method when 
doing a history study. 

When historians, especially intellectual historians, talk about theory and/or method of history they mean 
something different than educational technologists or instructional designers mean when they talk about theory and 
method in their area of specialty. 

Traditionally, although not exclusively, educational technologists think in terms of a method of research 
(research methodology) such as experimental design, as a way to investigate, test or prove a theory (or some aspect 
of it) of instruction or delivery. They see a clear difference between theory and method. The method is a way of 
gathering data in order to test the theory. It seems desirable to maintain this difference. 

Historians do not seek to maintain a difference between theory and method but they so seem to acknowledge 
that there is such a distinction. Analysis and discussions about theory and method tend to occur within the context 
of a particular historical study. In the literature of the field of educational technology theory and methodological 
considerations are usually identified at the beginning of a piece of research. 

Historians seem to view the relationship of theory to method much like researchers doing participant- 
observation or qualitative studies do. There is not the emphasis or effort to separate the knower (researcher), the 
known (object of study), and the method (the data gathering and analysis techniques) as there might be in traditional 
educational technology research. In fact, historical research is often considered to be qualitative research. Although I 
am more comfortable thinking that historical research is a blend of qualitative research methodology (interviews, 
document search) and conceptual research (approaches from philosophy such as metaphor and concept analysis). 

There are specific forums for discussions of theory and method in journals such as The Journal of the 
History of Ideas and History and Theory. Time and space (and my lack of in depth knowledge) will simply not 
allow for a thorough analysis here. 

Studying conflict: The historiographical problem of educational technology 

Intellectual history can be viewed as an interaction. It can be the interaction between the empirical and the 
hermeneutical (the intent to understand), the interaction between science and art, the interaction between analysis and 
expression, or even the interaction between events and thoughts. As used here interaction means a sort of give and 
take. Sometimes that give and take becomes a conflict. Intellectual history can even be viewed as the study of 
intellectual conflict. A problem of writing an intellectual history of an academic field was described by Ludwik 



It is very difficult, if not impossible, to give an accurate historical account of an academic discipline or a 
field of study. Many developing strands of thought intersect and interact with one another. All of these 
would have to be represented; first, as continuous lines of development and, second, in everyone of their 
many intersections and connections. Third, the main direction of the development, taken as an 'idealized 
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average’, would have to be described separately and at the same time. The continuity of the line of thought 
that has already been mapped out must continually be interrupted to introduce other lines of thought. The 
main current of thought would often have to be held up in order to investigate and explain any connections. 
Often, much has to be omitted to preserve the main current. Instead of a description of dynamic interactions 
one is often left with an artificial and arbitrary scheme (Fleck, 1979, p. 19). 

Fleck is describing the difficulty in developing an historical document that follows a chronological order yet 
maintains a flow which keeps the reader’s attention. He admits that many things are happening simultaneously. 
Ultimately, it is up to the judgement of the individual historian and writer to determine what content is included in 
the study and what the sequence of presentation will be. 

I think it is also important to think of Fleck’s statement as it relates to the study of conflict and intellectual 
history of an academic field. The historian must ask ’’what should I include in this analysis?"; "what is the 'main 
current’ of thought here?"; "how did it become the main current of thought?" ; "what alternatives were rejected in 
order for this thought (or combination of thoughts) to become the main current?’’; etc., the latter questions open the 
door to studying conflict. 

John Wettersten (1975) identified two fundamental considerations of history and its methodological/ 
theoretical considerations in writing the history of psychology; (1) inductivist history, which includes "true 
theories", facts, the discoverers, and dates of discovery; and (2) conventionalist history, which says that the theories 
that were modified to get to present theories should be included. These considerations are also important to studies 
in the history of educational technology because many educational technologists see themselves as being closely 
related to and dependent on psychology and learning theory for theory and method. Wettersten puts the fundamental 
problem of writing a history of psychology this way: 

Though historians of psychology have attempted to meet both the inductivist and conventionalist standards, 
a successful history of psychology cannot be written in this way. There is one crucial fact which forces historians of 
psychology (of either bent) into difficulties: that the history of psychology is, for the most part, a history of 
schools. The research produced by the schools does not fit inductivist or conventionalist standards because the 
theories of different schools contradict each other... (Two) contradictory theories cannot both be true (theory or) be 
modified anticedents of contemporary theory (p. 157). 

Wettersten identified five techniques which were used by historians of psychology to avoid their 
fundamental problem. These techniques "are used to avoid the discussions of controversies, mistakes and problems, 
and this avoidance leads to a misleading picture of the history of scientific psychology" (pp. 157- 158). The five 
techniques are: (1) vague and uncritical praise of certain theories; (2) recognition of fact gathering regardless of the 
significance of the facts; (3) uncritical praise of methodology; (4) recognition of techniques regardless of the results 
they produce; and (5) discussion of careers of individuals. 

To some extent, attempts to write a history of educational technology face this same dilemma. These 
writers have also used the same techniques for avoiding that dilemna. Although it is certainly true that the nuances 
of practice more readily allow for the merging of differing theory bases. But there are probably more interpretations 
of the basic theories in an applied field which further complicates the problem. 

Wettersten argues that there are two solutions to the problems of the incompatibility of contemporary 
theories: "one may write the history of a single school, or one may seek to reconcile different schools and their 
histories" (p.159). In either case criticism and failure must be addressed as part of the history if it is not the focus of 
the history. 

His reasoning is that "the history of psychology consists for the most part of the development, conflict and 
decline of schools" (if a critical history is avoided) "the actual problems, theories aims and mistakes of 
psychologists- the most important events to understand if one wants to understand the history of any intellectual 
tradition-are omitted or distorted" (p. 171). It seems that in order to write a good intellectual history conflicts cannot 
be avoided. 

What is the root of conflict in an academic field (besides ego and personality issues)? People disagree about 
ideas, concepts and applications. There seem to be four main reasons for this disagreement: (1) different definitions 
of terms; (2) the use of different information or authorities; (3) the acceptance of different premises; (4) different 
inferences drawn from the same premises. 

Frequently, particular ideas or conceptions are introduced into professional dialogue for reasons of personal 
preference and belief (not quite personal reasons but close). Over time this motivation becomes hidden as particular 
ideas grow in their acceptance as do the plausible options. Historical investigations bring these hidden motivations 
and options to light. Intellectual historians of educational technology contribute to the self-awareness of the field. 
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They do this by helping to make the lost and hidden purposes into conscious ones. This result will open them to a 
critical appraisal that may rekindle the discussion of the moral and ethical responsibility of the professional in our 
field. 
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What I think I do and why I think I do it 

I teach concepts. Maybe not like Merrill and Tennyson (1977) suggest but I teach concepts nontheless. I 
suppose that most of you are thinking something like "big deal...I do too". I have no doubt that you do. I am sure 
that you teach concepts as part of instructional design (or even how to teach concepts), media utilization, computer 
applications, or whatever courses that you teach. But I think I do it differently than most of you do it. I suppose 
that most of you are thinking something like "well.. .we all teach differently." I agree. But there is more to it than 
selecting from a variety of sequencing or delivery strategies. I think that it is a general outlook on teaching. I think 
that I emphasize "teaching that" or "teaching about" rather than "teaching how" (Green, 1968). 

First let me say that I do not believe that I am a better person than others in the field of educational 
technology because I teach concepts and they may have some other focus. But teaching concepts and the 
accompanying outlook works for me and my students at several different levels and so I do it. I’m sure that others do 
what works for them. Teaching concepts is overall a fundamental approach to the field. 

I don’t think that I invented anything new. People have "taught concepts" in a variety of fields of study for 
years. I have witnessed people teaching concepts in the field of education for over twenty years. This includes 
people with whom I have taken courses from the field of educational technology (Syracuse University). So its not 
new. But I do have my own twist. 

I am comfortable teaching concepts. Here is a brief explanation why this is the case. In my ten and one 
half years as an undergraduate student I changed areas of concentration six to ten times (depending on how you count 
them) and often the only thing I was able to take with me from program to program were the concepts (not all the 
credits counted). Concepts like "good government" and "democracy" were/are as important to political scientists as 
they were to journalists, historians, and teachers. Since I majored in all of these areas at one time or another I made 
the best of the situation. 

The first Master of Science Degree I was awarded was in the area of the Cultural Foundations of Education 
from the University of Wisconsin-Milwaukee. This program had an interdisciplinary approach to the study of 
education (given the amount of majors that I had as an undergraduate I suppose I’m a natural for an interdisciplinary 
approach). It seemed that the "thing" that unified the disciplines (history, philosophy, sociology, anthropology) in 
an interdisciplinary approach was the concepts) that was being studied. I find that the concepts involved in 
educational technology are broader than, perhaps super ordinate to, any specific academic discipline which contributes 
to the knowledge base of the field (this means that the study of educational technology itself is interdisciplinary and 
the study of the concepts involved in educational technology is also interdisciplinary). 

After I entered the graduate program in Instructional Design, Development, and Evaluation at Syracuse 
University it did not take very long before I realized that the field of educational technology was rich in concepts and 
ripe for picking (actually, given the name of the program, if I had been paying attention I might have figured this 
out earlier). I decided fairly early in my graduate studies in educational technology that it was possible to do lots of 
scholarship in, on, and around its concepts. I thought that the field might be ready for serious study without 
studying more "how to's" which I considered rather boring. 

This was particularly gratifying since I was not thrilled by the prospect of conducting quantitative studies 
using quasi-experimental design techniques and/or inferential statistics. Its not that I still think that these kinds of 
studies are totally useless. They have and still can contribute to the development of some practical heuristics. But I 
do freely admit that I think that there can be no real "causal claims" made about our field (except perhaps that it 
sometimes causes me to wretch but even that's not guaranteed). I suppose that my distrust/dislike of quantitative 
methods in our field can also be attributed to some extent to my background in the cultural foundations of education. 

In the second year of my studies at Syracuse two events occurred that, while they happened independently of 
each other, when taken together had a lasting effect on my outlook toward the field. Especially my outlook on my 
teaching and my scholarship which I consider to be inextricably intertwined. 

The first of these events was a conversation that I was involved in with several of my graduate student 
colleagues (these students shall remain nameless because I can’t really remember who they all were, or even how 





many of us were there but the number five sticks out in my mind). The outcome of the conversation was that there 
were about six to ten primary concepts in our field. A list of these included analysis, development, design, 
formative and summative evaluation, management, systems, and systematic. When understood from this perspective 
we realized that everything else that happened in our practice was a part of one of these primary concepts, or it was 
an interpretation of one of these primary concepts, or it was an application of one of these primary concepts. It all 
seemed so incredibly base. We asked each other "is this all there is?" (I think that it is interesting that left to their 
own devices graduate students can so easily get to the root of supposedly complex ideas, or at least they think they 
do. Not just the group that I talked about there are many such instances that can be recounted at different 
institutions. And not that graduate students are always right when they do this but interesting simplifications can 
occur. More interesting is that when many of these very same graduate students become faculty members they 
contribute to the literature which further complicates things. I'll deal with this in another paper at another time ). 

The second of these events was a personal conversation that I had with either Don Ely or Phil Doughty (I'm 
inclined to think it was Ely because I could more easily imagine myself having this conversation with him but I 
can’t rule Doughty out with certainty). The gist of this conversation was that while it is true that definitions of 
educational technology/instructional technology have changed fairly frequently (historically speaking) in the past 
thirty years or so the concepts that make up those definitions or that are included in those definitions have not really 
changed. What seems to have happened was that while there has been a substantial movement from a "tools" 
conception of educational technology to a "systems" conception of educational technology the concepts that are used 
to discuss those differing conceptions stayed the same. But, the interpretations of those concepts (and subsequent 
practice involving those concepts) are very different in a tools conception of educational technology and a systems 
conception of educational technology. I took this to mean that if one wanted to get a good understanding of 
educational technology the concepts were ultimately more important than their applications (procedures). 

What this has meant for my teaching in general and for my teaching about educational technology in 
particular is that I focus on the primary concepts because they have been around a long time and they are at the root 
of most of what goes on in the field. I think that is most important for students of educational technology to have a 
solid understanding of those primary concepts. As part of that understanding of the primary concepts I try to make 
sure that my students know that there are a number of different interpretations of each of the primary concepts and 
that professional practice will "play out" differently undereach of those different interpretations. I think that this is 
particularly important for me because about sixty percent of my students are K-12 teachers and the remaining forty 
percent work, or are interested in working, in higher education, health education, the military the not for profit 
sector, or want to go on to do further graduate study. They will use the concepts differently. They will have 
different meanings. 

This can be summed up in the following way: ’there is more than one right way to "do" educational 
technology. But this does not mean that "anything goes". There are wrong ways to do it as well’. 

I have three brief comments in closing this section. First, if as Lakoff and Johnson suggest that concepts 
direct our thought (Lakoff and Johnson, 1980). Then it seems reasonable to assert that teaching concepts is not just 
an intellectual exercise. If concepts direct thought and if we think logically/rationally before we act (a basic premise 
of educational technology) then it is safe to claim that concepts and conceptual structures contribute greatly to our 
professional practice. So I feel justified in using the approach that I do. 

Second, I try to help my students get at the meaning of concepts using a combination of four approaches to 
studying/analyzing concepts. These four approaches are: 1) a necessary and sufficient conditions approach. This 
approach dates back to Aristotle. The object is to describe the critical characteristics of a concept and to determine 
whether or not a particular instance (or example) is a member of a larger set (the larger set being the concept); 2) 
conceptual analysis. This has its history in analytic and linguistic philosophy of the twentieth century. The object 
here is to determine meaning through intent and use of a concept; 3) a paired opposites approach. The object here is 
to analyze two concepts simultaneously to see how they are the same and how they differ. Akin to a compare and 
contrast essay exam; and 4) metaphors. There are two approaches to metaphor (and I won’t do either of them justice 
here). The first is to try to explain one concept in terms of another that is not immediately apparent. The object 
here is to provide an intentional comparison. The second is root metaphor. The object here is to analyze hidden 
meanings and implications through the way a particular concept is talked about. 

Finally, I guess that this also has some implications for my teaching. It might be called "constructivist" or 
"anchored" or "situated" because of the strategies that I use to teach, the project structure that I use to assess student 
knowledge/progress, and because of the reasonably clean flow between my teaching and my student's projects. I 
make no such claim. Not because I think that these are bad ideas but because I don't think that I have a good 
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understanding of these ideas (at least I’m not satisfied with my ability to articulate my understanding of them). I 
have too much uncertainty about the basic categories or conceptions of constructivism, as epistemology, as a theory 
of learning, and as instructional theory. 

An in class activity 

In the third session of my course GRED 653 The Instructional Development and Planning Process we 
discuss Instructional Development Models (models might also be considered a primary concept of educational 
technology). This is a way of getting at the idea of the entire instructional development process and relating the 
instructional development process to problem solving, problem solving models, and problem solving strategies (the 
subject matter of session two). For this session the students will have read Kent Gustafson’s book Survey of 
Instructional Development Models (1991 edition), a chapter from Cass Gentry’s book Introduction to Instructional 
Development (1994), and a series of handouts including some definitions, diagrams, and lecture note organizers (not 
that I follow this lecture note organizer mind you, but it does give students the initial idea that I am well organized 
even if I don’t always come off that way). 

After some initial discussion about models in general and instructional development models in particular, I 
ask students to take a few minutes and write a description or definition of instructional development. When they are 
through I ask them to read their responses and I write these on the board (sometimes I use an overhead but usually 
the board because it can add to my theatrics). It doesn’t take long before the board space is full and I have to begin 
focusing on words which are repeated and I circle or underline these and I add new and interesting phrases that are 
suggested as well. 

The "lecture/discussion/analysis" that I do along the way (while they are reading their descriptions, more 
accurately in between their responses) is made up of two parts that occur at the same time. The first part is in 
response to the concepts that the students suggest that denote the activities included in instructional development 
(e.g.. analyze, design, manage, etc.). The other major focus of my "presentation” comes out of the words that the 
students use to describe the overall action of instructional development (e.g.. id is a system that..., id is a method 
for..., id is a process by which. ..etc.). 

It doesn’t take long before the students generally agree that there are five generic activities or stages in the 
instructional development process; analysis, design, production/prototype/development (this seems to be the 
toughest to get a general agreement on), implementation, and evaluation. It is further understood that there are 
different specific concepts, tasks, procedures, and ideas that can occur during the different stages. 

The next activity involves an analysis of the concepts that are used to depict the overall instructional 
development process. I divide the class into small discussion groups and assign each of the groups the task of 
ranking some concepts as to the rigidity that they denote, a rigidity/flexibility scale if you will. Included in the list 
of concepts such as: approach, plan, method, process, procedure, system, direction, formula, framework, algorithm, 
recipe, blueprint, pattern, technique, heuristic, prescription, etc.,. Many of these come from the students 
descriptions and some I have to work in as well. In addition to ranking these concepts the students also explain their 
reasoning as to why they ranked the concepts as they did. 

After students present their rankings and rationales and a general comparison/analysis occurs I have them to 
do one last activity as individuals. I ask them (again) to write a brief description of instructional development. After 
they have completed that task (and this is the big finish for me) I ask them to spend a few minutes thinking about 
their descriptions (as individuals) while they substitute or "plug in’’ each of the concepts that had ranked to 
characterize instructional development. After they have had a few minutes to do this I ask which of the concepts do 
they like/dislike and which of them completely change the meaning intended by their description/definition. A good 
discussion regarding the importance of precision and comfort with concepts has always followed. 

There are several things that are important to note about the five stages in instructional development. Like 
the stages in John Dewey’s "complete act of thought" (1933) I argue that these should not be thought of as steps in a 
procedure because this implies a certain linearity which is seldom if ever the case in real life. The fact of the matter 
is that we usually move from one stage to the next but we also jump around as new information becomes available 
for us to consider. Hence we can always be (and likely are) rethinking what we have done in light of new 
circumstances. We do this in a fluid fashion and not as stiffly as is implied by graphic representations in 
instructional development models 

The bottom line for me is "comfort" with the concepts and conceptions of ideas (both what they mean and 
what they imply). This means that students have to know what and agree with what concepts represent and they 
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have to be comfortable with the label itself. Memorizing and adopting others conceptions is not good enough. If 
students aren't comfortable with their conceptions of ideas then it is much less likely that they will put these ideas 
into action in their professional lives. Perhaps even worse, if they are really ill at ease with particular concepts then 
they may work to resist particular actions associated with them. Its simply a matter of diffusion, adoption, or 
perhaps better yet, adaptation. 
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Theoretical Perspectives 

Leaders in education, business, research and government have emphasized the need to rethink our goals for 
education. Concurrent with the redefinition of educational goals is the emergence of new assumptions about learning 
and instruction. Schools are responding by creating information-rich environments for instruction and administration 
utilizing technology as the vehicle for restructuring education to meet the needs and challenges of our information 
society. 

These rich environments promote active knowledge construction in authentic and meaningful contexts. 
They also encourage students to assume a more responsible role in their own learning (Grabinger & Dunlap, 1994); 
support the development of collaborative decision-making and problem-solving; and foster the development of 
research and meta-cognitive skills. Social interaction is an important component in these environments, supporting 
cognitive development (Vygotsky, 1987; Wertsch, 1985). This approach stresses the process of learning, including 
the important component of reflection, rather than the learning of content alone. This philosophy is consistent with 
preparing students to work and live in a technological society. 

Project-based (Honebein, Duffy, & Fishman; 1993) and problem-based learning (Savery & Duffy, 1994) 
are instructional methods which are being used to promote active and authentic learning. A graduate education course 
was first implemented using a project-based approach. The second time the course was offered, problem-based 
learning (PBL) using Barrows (1988) tutorial process was used. 

Research Questions 

The questions which focused this study included: 

I. What are the differences in: 

1 . Student Outcomes 

a. knowledge, skills, process 

b. attitude 

2. Classroom strategies/interactions 

when using project-based versus problem-based learning 
in the classroom. 

II. What types of objectives does each method support? 

ID. How is student collaboration supported by each methodology? 

IV. How did technology support each methodology? 

V. What issues arise when using these methodologies in graduate 
courses? 

Methods 

The participants were graduate education students enrolled in an elective course. The course met once a 
week for 4 hours and 40 minutes, over a quarter. One class met in a College of Education computer lab. The PBL 
class met in the Instructional Resource Center. 
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The first quarter the course was taught using an inquiry-oriented, project-based approach with students 
working in collaborative groups. The class worked on three main projects with individual groups choosing the focus 
for their inquiry. The projects included the investigation of curricular frameworks and educational goals; research on 
both new forms of assessment compatible with evolving educational goals and learning theory, and ways that 
technology can support assessment; and the investigation of current and emerging uses of technology within an area 
of interest. In addition, students critiqued learning activities that had a technology component, and educational 
software. Students were presented with a choice from four possible final projects or they could propose an 
alternative project of their own. 

Students critiqued their own group’s process and other groups’ projects. Questionnaires were used, not only 
to collect data on process and for purposes of grading, but to also support student reflection. All of the data was 
electronically collected through the use of hypermedia, word processing, or email. Students were required to keep 
both a journal and a portfolio. 

The course was also conducted using a problem-based learning (PBL) approach using Barrows (1988) 
tutorial process. A HyperCard stack was developed to record student information including their backgrounds, 
interests, and views on learning and technology. The tutorial sessions were first offered using flip charts. Software 
tools were then designed to help facilitate the tutorial sessions, students' self-directed study, and the facilitator’s role. 

The class worked on three problems during the quarter, two with the class and one in small groups. The 
students used email and the internet extensively. They shared and critiqued resources, kept journals, and evaluated 
their own problem-solving, self-directed learning, and support of the group process. In addition students critiqued a 
technology-supported learning activity. Students were again given a choice of final projects, and could choose 
whether to work individually or in groups. 

Data was collected through observation, questionnaires, documents such as journals, self and group 
evaluations and email, and artifacts (student projects and portfolios). QSR NUD*IST was used to facilitate the 
analysis of the data. 

Results 

Student Outcomes 

Students' views of their learning was evidenced in their journals, self-evaluations, and exit surveys. 
Students in both classes were at first uncomfortable, particularly concerning what was expected of them (their 
role/responsibilities) and how they would be evaluated. They all stated that they ’’learned a lot.” 

Student reflections on group interactions and participation were very consistent with the overall class 
activity. In the project-based class, students discussed their participation and group interactions in product-oriented 
terms. In the PBL class, students discussed their contributions in terms of ideas, information shared, and 
communication. 

Students in the PBL class were instructed to email their self-evaluations to the professor, which were to 
include how they thought they did as a problem solver, as a self-directed learner, and how they might work differently 
the next time. Students reflected on the benefits of the problem-solving process that they were learning, and on 
problem-solving within a group. After the first problem, one student described a benefit of working as a member of 
a group: 



I felt that I gained many insights into how others would solve the same problem. That is very 
valuable when trying to force yourself to see things from another perspective because in most 
instances there is more than one right answer. 

Another student described a benefit of working in groups over ’’traditional, teacher-based lecturing 
techniques”: 

The opportunity for peer-taught input can really enhance the amount of information that can be 
passed along and assimilated. 
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An additional benefit of group work was described: 



I like collaboration and getting input from different people. It helps keep an open mind and make 
me think in various ways I might not have thought of before. 

In evaluating their participation as a group member, a number of students in the PBL group mentioned their 
own communication skills and learning styles. Some students discussed that they felt "some initial anxiety" and 
that they were intimidated by group members who seemed to have more knowledge and experience that was related to 
the course. 

A benefit of solving the problems that was described by students was learning to "understand the implied 
problem" and "deciding when I had enough information to answer the question." 

Students discussed their attitude toward the course methodology. This included the problem solving 
process, relevance of the problems, and inquiry skills. Students stated that they experienced satisfaction in working 
through the problems. One student stated: 

The learning method we used ... is, to me, very intuitive. It is the way we learn things in the 
"real world." From language to a job task, when we become active participants in the context of 
that which we seek to learn, it becomes a part of us and not a memorized appendage to be 
regurgitated for an exam and discarded immediately thereafter. I like the cross-discipline nature of 
this learning process. 



Classroom Strategies/Interactions 

Students in the project-based course spent more time learning the software. Meeting only once a week to 
work together with the technology, limits the experiences and benefits of this model. The experience is different 
from the K- 12 project-based classroom that meets daily. Students required that most of the class time be spent using 
the technology to complete projects. Time for discussion was, therefore, limited. 

Group process was studied. The group work that was done in the project-based class was at first, 
collaborative in nature as students discussed possible topics, posed questions to one another, and made decisions 
concerning the tasks that they needed to accomplish. Some of the initial data gathering was also done in a 
collaborative manner. As the projects progressed, students began feeling that there wasn't enough time and some 
group projects were pieced together individual projects rather than true “collaborative” work. In addition, when given 
the choice of group or individual final projects, students chose to do individual projects. The resources that students 
used for their inquiry was very limited compared to the resources used by the PBL class. Students chose a narrow 
focus for their group projects. The driving goal was on producing a product rather than inquiry and learning. 

In the PBL class the benefits of collaborating became apparent to students as they learned and experienced 
the process. Collaboration continued throughout the quarter. 

The PBL class had more opportunity to discuss, apply, and synthesize the information contained in class 
readings and in the additional resources that they found and shared. The PBL process also supported inquiry better in 
terms of scaffolding the process and time (it is the focus of what students are expected to do between meetings). 

The over-riding goal in the PBL-class was inquiry. Students not only learned about and used more 
resources, they also critiqued and shared resources with other students. The problem environment supported in-depth 
discussion of relevant topics and revisiting concepts and their relationships. The problems overlapped and built on 
each other. When given the choice of group or individual final projects, students in the PBL class chose to do group 
projects. 

Both the projects and problems were designed so that students would “discover” important ideas on their 
own. This was done instead of “telling”, in order to support conceptual change. Students enjoyed and learned in 
both classes but the students in the PBL environment displayed more ongoing and consistent motivation for both the 
class time and inquiry outside of class. 

Objectives Supported 

This class was originally designed to support conceptual change. The course design was consistent with a 
constructivist view of learning, new educational goals, and a view of the role of technology as a tool that can 
support the attainment of educational goals and increased student learning. The project-based class was designed to 
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model K-12 active learning environments where technology is used to support inquiry, to share group projects, and 
to support student motivation. The PBL class utilized Barrows tutorial method (Barrows, 1988). 

Collaboration was an important aspect of these learning environments. The objectives of problem-based 
learning include developing and applying the following lifelong learning skills: 

• reason through problems 

• identify learning issues 

• identify and use appropriate information resources 

• support group process 

• ability to evaluate the performance of self and others in a positive and constructive manner. 

The students in the PBL environment both chose to collaborate and reflected on the benefits of collaboration more 
than students in the project-based class. This seemed to be in part due to the amount of time needed with the 
software and difficulties collaborating over a distance, on technology-based projects. 

In the PBL course the requirement of self-evaluation supported the reflection necessary for metacognition 
and individual development as problem-solvers and independent learners. 

With these students, the PBL problems and course structure both encouraged more extensive inquiry and 
appeared to provide more motivation than the projects. 

Both classes were structured so that students would need to explore the same issues. The structure of the 
PBL class provided more group input and discussion and for a complex and interconnected domain, it scaffolded the 
required cognitive processes, better than the project-based learning approach. 

Technology 

Students used technology for communication, inquiry, and productivity. The PBL class used a facilitating 
stack with problems two and three. Students discussed the role of technology. They described both the benefits of 
electronic searches and frustrating experiences with software and the internet. The PBL group worked on problem 
three in small groups using a hypercard stack which was designed to facilitate the process. The reaction to the 
software was very positive. Students described its benefits as providing structure and organization and facilitating 
group participation. One student also noted: 

when using the software for these purposes, the computer becomes the focal point of the group. ... 

The computer almost becomes a facilitator in itself. 



Issues 

Students have "learned" traditional roles and feel anxious when they are not sure of what is expected of them 
and how they will be evaluated. In the PBL class, students were more confident and felt more successful as they 
progressed. Both understanding the process and having some additional background knowledge were cited as 
contributing factors. Students also preferred working in smaller groups with problem three. They realized, however, 
that their previous experience learning the process and the knowledge they acquired working on the first two 
problems, played a major role in their success and enjoyment with the third problem. 

Time and schedules were also issues with both of these methodologies. The type of group work required of 
the project-based class also had the additional issue of technology access. 

Inquiry was an important component of both classes. The amount of information, its accessibility, and 
evaluating sources, were issues that surfaced. Students are becoming overwhelmed by the amount of information 
that they somehow need to wade through. 

Educational Implications 

This study provides rich data which supports the use of problem-based learning in the higher education 
classroom. It also provides factors to consider when designing active learning environments, comparing the different 
types of objectives that are best met by either PBL or project-based learning. 
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Abstract 

Cases have been recommended as an important instructional tool for developing professional knowledge 
across disciplines. In this paper , we report on the design and use of a Web-based instructional design case in a team 
case competition involving six universities. Students and most officials were enthusiastic about the use of ID cases 
and about this event. Team collaboration and competition were noted as motivating factors for students. The 
findings were used to inform on-going research and development , which is also described. 

"Needs analysis! Why should we want a needs analysis? We already know what we want to do!" Five heads 
nodded in agreement as I looked around the table. I tried to read the expressions on the faces of the members of the 
Workplace Readiness Project Committee: irritation?, speculation? boredom? hostility? This was my first meeting 
with the committee and my hopes for it going well were rapidly collapsing.” 

So begins a case on instructional design, "The Trials of Terry Kirkland" (Hrabe, Larsen, & Kinzie, 1996). 
In the case, a novice instructional designer comes up against thorny professional practice issues for which she was 
not prepared. For students analyzing cases such as this one, the cases provide an opportunity to explore professional 
issues while the students are still learning about design. Even when ID training includes applied design projects and 
reflection on relevant theories and techniques, the use of cases can ensure a more comprehensive preparation: a 
greater number of design issues are explored, in a broader array of environments, than would otherwise be 
encountered. 

Building on the growing popularity of cases within education, and following recommendations by Graf 
(1991) and Ertmer and Russell (1995), we have been using case methods within instructional design classes 
(Lindeman, et al., 1995; Kinzie, Larsen, & Kent, 1996). Our most recent efforts involve development of ID cases 
and the use of the World-Wide Web (or Web, for short) as a delivery medium, enabling use of the materials by 
students at any institution or by any interested individual. Further, we are exploring the combination of team 
collaboration and team competition during the case analysis process. This paper will report on a team case event 
held during the spring of 1996 with six universities across the United States. 

We begin by providing background information on case methods and techniques used to help students learn 
about instructional design practice. Then we describe the 1996 ID Team Case Competition, which was implemented 
with teams from six instructional technology programs across the United States. The competition case is described 
and the event methods detailed. Evaluative data are also presented, based on a follow-up survey of participants and 
event officials. We close with observations about the potential value of case methods and case events, and provide 
recommendations for future development. 

Case Methods 

Cases are used extensively for professional preparation in law, medicine, and business, but have only 
recently begun to be widely employed in education (Merseth, 1991). Cases can involve a description of real events, 
as is common in business cases that describe the processes at work in an actual corporation. They can also resemble 
story-based fiction that is written around a central theme or set of key issues but which is grounded in problems and 
challenges from the real world (Ertmer & Russell, 1995). Instructional cases are used to encourage the development 
of professional thinking, as individuals formulate reactions to case materials. 

Case methodology is especially effective if students are required to identify facts and issues, to de-center and 
view events from different perspectives, to apply current professional knowledge and research, and to predict 
consequences of various courses of action (McNergney, Herbert, and Ford, 1993). In this way, the use of case 
methods can help students to forge important connections between the academic and the experiential, between 
knowledge and practice (Cooper and McNergney, 1995). The effectiveness of case-based teaching is supported by 
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Kleinfeld (1989, 1991), who has demonstrated that teaching with cases helps students to understand the meaning of 
events, increase their ability to frame educational problems, and improve their thinking regarding alternative courses 
of action. 

Learning about Instructional Design through Case Methods 

Instructional Technology (IT) majors typically learn instructional theories and design models and use them 
to guide their instructional design and development. Frequently students employ these techniques in an artificial 
environment, however, without an authentic instructional need to drive their efforts. There is no client to work with 
and no real-world problem to explore. On the other hand, securing the involvement of actual clients is difficult when 
the design consultants are students. Case methods are particularly useful in this situation, as they provide an 
environment in which students can explore a real problem, attempt to understand it, and then consider and generate a 
response. We do not consider case methods to be a substitute for applied project experience, however it is a valuable 
supplement that can add breadth and depth to students' knowledge. 

We advocate consideration of cases by teams of students, due to the benefits realized through collaboration 
and because professional practice within instructional design most often requires individuals to function effectively 
and creatively in a problem-solving team. This strategy has proven effective within previous team case events, 
where a case scenario provided a rare opportunity for professional collaboration on solving real-life problems (Kent, 
Herbert, & McNergney, 1995). Ellsworth (1994) explains that collaborating students take on a more active role in 
the learning process. They become problem-solvers, contributors and discussants. The process of team 
collaboration can enhance the case experience, providing multiple points of view and offering individuals the 
opportunity to advance, and develop support for, their own perspectives. 

We have combined team collaboration with inter-team competition. This approach is similar to the pairing 
of cooperative and competitive strategies advanced by Johnson and Johnson (1994), who suggest that this 
combination can be effective when the focus is on well-learned skills that need to be practiced (such as, in this case, 
applying ID skills to a novel case situation). Our collaboration/competition model is adapted from that advanced by 
Kent, Herbert, and McNergney (1995). Kent and his colleagues have asserted that competition can help ensure rigor 
in education, particularly if judges render opinions on team performance that is linked to pre-established criteria: 

"Setting performance standards and using such measures to gauge students' behaviors encourages 
programmatic rigor in education just as these activities do in other professional fields" (p. 139). 

The competition aspects of the case experience allow this activity to reflect the real world, where a design 
team must sometimes compete with others to identify the best possible solution. We also feel that students bring 
an energy and focus to their team collaboration that might not be present without the element of competition, as 
students know that their team's performance will be evaluated alongside that of other teams. It is our opinion that 
competition can be a useful adjunct to collaboration, provided that the primary focus is on learning, not on winning. 

Case Media 

The first case format proposed to the education community was the print medium, which continues to be 
the most popular form (Shulman, 1987). Internet technologies, however, have provided new vehicles for delivering 
cases to learners. 

We have devoted some previous efforts (Lindeman, et al., 1994; Kinzie, Larsen, & Kent, 1996) to 
exploring the use of the Internet to provide both case materials and on-line environments for case discussions. We 
began (Lindeman, et al., 1994) with the use of MOOs, an acronym for "MUD, Object-Oriented." A MUD is a 
Multi-User Dimension, an on-line environment peopled by users who synchronously interact with one another. In a 
MOO environment, "text objects" are created and left for users to find, read, and discuss. Our first goals involved 
creating an explorable professional practice environment, such as a suite of offices containing documents in filing 
cabinets, organization charts on the walls, and transcripts of meetings that could be "played back" (the text appears 
and scrolls up the screen during playback). While it was an interesting idea and one we may return to later, we found 
that, without experience and comfort in the MOO environment, case materials were too difficult for students to 
access and discuss. 

In our next effort (Kinzie, Larsen, & Kent, 1996), we moved case materials to the World-Wide Web and 
kept the case discussion on-line in the MOO. We found the Web well-suited to case delivery, and providing graphic, 
sound, and video media, in addition to text. Being able to open Web documents alongside the MOO discussion 
window helped students manage and discuss the materials. The MOO environment continued to present challenges, 
however, since it allows multiple threads of conversation to occur simultaneously in real time, a feature some 
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students found frustrating and others found fascinating. We are interested in returning to this combination in the 
future, for we feel there is important potential for allowing geographically disparate students an opportunity to meet 
and discuss cases on-line. 

In the research reported here, we combined Web delivery of an instructional case with on-site team case 
meetings for discussion and response development. We also introduced the element of team competition along with 
team collaboration. And, perhaps most importantly, we invited others from the academic and professional 
community to participate. The primary question we asked was, "Are cases a worthwhile medium for exploring and 
learning about instructional design?" We were also interested in whether participants found team collaboration and 
competition to be valuable, whether the Web was an effective delivery medium, and whether our approach to case 
development resulted in realistic cases sufficiently deep for encouraging exploration. 

Methods 

Participants 

Teams participating in the 1996 competition were from the following institutions: Arizona State 

University, Pennsylvania State University, and the universities of Colorado-Denver, Minnesota, Southern Alabama, 
and Virginia. A total of 36 students participated, (20 female and 16 male). The students were from both master’s 
and doctoral programs, and all had had some formal training in instructional design as part of their respective 
programs. Two of the teams participated as part of a course; for the other four teams participation was an extra- 
curricular activity. On the average, student participants reported having a significant amount of full-time work 
experience (between 5 and 10 years). They possessed a broad range of experience from a variety of professions, 
including teaching, career military, and corporate. 

Officials included team sponsors and the provocateurs and judges nominated by each sponsor (each sponsor 
nominated one or more professionals for participation). Sponsors also nominated the student teams and relayed all 
event communications to team members. Provocateurs read team responses and composed a specific question for 
each team and a common question for all teams. Judges reviewed teams’ case and question responses and completed a 
rating scale and written comments for each team. 

Materials 

"The Trials of Terry Kirkland” was developed for the 1996 team case competition (Hrabe, Larsen, & Kinzie, 
1996). While fictional, this case is based on real issues and problems selected in advance by the case authors and 
imbued with actual experiences. The bulk of the case is delivered in an illustrated narrative, ostensibly written by an 
instructional designer. The relatively inexperienced designer has been brought into a high school to work with a 
group of teachers to develop a "workplace readiness” workshop. Events in the case are presented in a number of 
scenes that take place over the course of about five months. 




Figures 1-3: Screen Captures from the Web-based case, " The Trials of Terry Kirkland ** 

The narrative is supplemented with a collection of twelve case ancillaries: text documents, charts, 

photographs, and audio and video clips. These ancillaries help to depict and add depth to the case events. Because we 
were concerned about whether participants would be able to access the audio and video clips, however, transcripts 
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were provided for these materials in these media. The case may be examined at the following URL: 
http://teach.virginia.edu/go/TTcases 




Figures 4-6: Ancillary case documents, audio, and video 

Procedures: 

Teams were given two weeks to review the case, discuss it, and develop their response. A limit of six 
hours was placed on team meeting time, though no limit was placed on individual reading, thought, or writing. 
Teams were allowed to refer to any resource materials they desired, but were instructed to respond to the case without 
the participation of their faculty sponsors. 

In developing their responses, teams were instructed to address each of the following tasks: 

• Identify the key issues present in the case, 

• Consider the issues from different perspectives, including those of the key players in the case, 

• Identify what professional knowledge team members have that would be pertinent (and what more they need to 
know), 

• Develop a plan of action, picking up at the conclusion of the case, and 

• Hypothesize as to the possible outcomes of that plan. 

Discussion of each of the above was limited to 250 words. The entire case response was required to be 1 ,250 words 
or less. 

Following submission of their case responses, teams were sent two questions from event provocateurs (a 
team of three experts served as provocateurs). One question was a general case-related question, and the second was 
developed in reaction to each team's analysis. Teams were allowed up to two hours (within a one-week period) to 
discuss and develop their response to both of the provocateur questions. Teams' question responses, along with their 
initial case response, were then sent to the five-member panel of judges. Individually, each judge reviewed the 
materials from each team, completed a rating scale for that team, and wrote evaluative feedback for the team. On 
tabulation of the judges' ratings, two winners were announced and their responses posted to the Web site. 

Measures 

Judges completed a rating scale to indicate the success with which each team addressed the five categories of 
case response and the issues raised in the provocateur questions. Table 1 contains a listing of these questions. 
Response was made using a four-point scale, with response options ranging from 1 (Strongly Disagree) to 4 
(Strongly Agree). 
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Table 1: Judge Rating Scale for Team Case Responses 

1 . The overall performance of the team was excellent. 

2. The team identified all of the important issues in the case. 

3. The team demonstrated an excellent ability to define relevant perspectives (e.g., 
instructional designer, teachers, students, administrators, community members, etc.) 

4. The team demonstrated appropriate application of professional knowledge. 

5. The team’s projected actions were reasonable and appropriate. 

6. The team effectively anticipated the consequences of actions. 

7. The team’s response to the COMMON question addressed the relevant issues and 
demonstrated insight into professional practice. 

8. The team’s response to the SPECIFIC question addressed the relevant issues and 
demonstrated insight into professional practice. 



When participants (Students, Sponsors, Provocateurs, and Judges) had completed their participation in the 
case event, we asked them to evaluate the experience by responding to a survey. Students responded to a survey 
made available for them on the Web; their responses were sent to us electronically through the use of Web forms. 
E-mail surveys were sent to Team Sponsors, Provocateurs, and Judges. While some of the survey questions varied 
according to type of participation, all participants were asked a common set of questions, which are displayed in 
Table 2. 
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Table 2: Survey Questions Answered By All Participants 



Survey Question 


Response Type 


Prior to this event, had you ever used cases as a learning tool? 

If so, what was the content (ID, teacher education, law, medicine) and how 
did you use the case(s? 


yes/no 
fill in 


How did you prepare for the case competition? 

Did you review the Web site? 

Did you read the practice case? 

Did you read any articles or other literature related to case methods? 
Others? Please describe. 


yes/no 
yes/no 
yes/no 
fill in 


How did you access the case materials? 

(Completely on-line, Only with print-outs, and Both on-line & printouts) 


mult, choice 


Did you download and watch/listen to the video and audio clips? 
(All, Most, Some, None) 


mult choice 


Did you have any difficulties accessing the case materials? 
If so, please describe. 


yes/no 
fill in 


Which of the ancillaries below did you feel were necessary to your understanding of the 
case? 

(12 ancillaries listed) 


checkboxes 


The use of the case study method is valuable in developing expertise related to 
instructional design. 

Strongly Agree, Agree, Disagree, Strongly Disagree 


Likert scale 


Participation in this case competition will help prepare students (helped prepare me) for 
future instructional design projects. 

Strongly Agree, Agree, Disagree, Strongly Disagree 


Likert scale 


Taking this experience as a whole, what worked? (or What was most valuable?) 


fill in 


What didn't work? (or What was least valuable?) 


fill in 


Do you have any suggestions for future modifications of this event? 


fill in 



In addition, students were asked to indicate the number of years they had held a full-time job (1-2 years, 2-5 
years, 5-10 years, or more than 10 years). Students and Team Sponsors were also asked whether their team 
participated in the event for a class or for some other form of academic credit, and whether there were other factors 
that prompted their participation. Responses to this survey were analyzed using simple descriptive statistics (means 
and standard deviations) and simple qualitative analysis for the open-ended questions. 

We also conducted 30-45 minute telephone interviews with participant volunteers after completion of the 
surveys. These interviews were tape recorded with permission and were later transcribed for analysis. Interviewees 
were all asked a series of standard questions but were encouraged to elaborate on their ideas and add any comments at 
will. Table 3 displays the interview questions we asked students and officials alike. Table 4 includes interview 
questions addressed to students only, while Table 5 contains questions directed to provocateurs and judges. In 
addition, we asked team sponsors whether this was a useful activity for their students to be involved in, and asked 
judges how they went about evaluating case responses. We use this data to more fully describe the effects of the case 
event. 

Table 3: Interview Questions Answered By All Participants 



• What other activities have you engaged in to leam/help others learn the practice of instructional 

design? ' 

• Was the case realistic? If so, what contributed to the realism? The story? The media 

components? The supporting files? 

• Did the media (graphics, video, audio) contribute something to the experience (over text alone)? 

What was that contribution? 



Table 4: Interview Questions Answered by Students 



• How did you organize your team's approach to the case analysis? 

• How did you assign responsibilities among team members? 

• How did you coordinate case analysis and response generation? 

• How many meetings did you have, and how long were they? 

• What kinds of discussions did your team have? 

• How did you deal with conflicting viewpoints within the team? 

• How did you feel about your case response? 

• Did knowing that it was going to be judged influence your approach to this activity? 

• How did you feel about the questions you received from the provocateurs? 

• How did you feel about the feedback you got from the judges? 



Table 5: Interview Questions Answered by Provocateurs and Judges 



• How did you feel about the quality of the case responses? 

• What kind of sense were you able to get of each team's instructional design expertise, based upon 

their case and question responses? 

• How were the teams' responses different from one another? 



Results 

Response Rates 

We received survey responses from 21 out of the 36 students initially participating, with at least 4 students 
dropping out, suggesting a response rate of at least 65%. (Two of the teams volunteered information on drop-out to 
us. We did not ask other teams if any members had been unable to participate, so are unable to be more specific.) 
At least one student from each team responded. Surveys were received from 9 of the 12 event officials, yielding a 
response rate of 75%. 

Following completion of the surveys, 12 students (38%) agreed to be interviewed, from five of the six 
teams, while 8 of the 12 event officials (67%) participated in interviews. 

Reasons for Participation 

The reasons participants gave for participating were varied: Seven students commented on their desire to 
learn more about instructional design through the case event and three noted that the competition aspects were very 
motivating-they were proud to represent their schools and reported giving team efforts high quality attention. One 
student wrote, "It seemed like a worthwhile adventure and it certainly exceeded my expectations." 

Preparation for the Event 

All of the students and officials responding indicated that they had reviewed the Web site and the practice 
case that had been made available prior to the release of the event case. Further, the practice case had been discussed 
by 62% of the students. Readings relating to case methods had been completed by 67% of the students and 33% of 
the officials. Cases had been used previously by 43% of the students and 78% of the event officials; however only 
one of the students and four of the officials reported experience with cases on instructional design. 

Use of Case Materials 

The competition case was reviewed both on-line and with print-outs by 95% of the students and 88% of the 
officials (however many students indicated in the follow-up interviews that the bulk of their case work was done with 
print-outs). Overall, students made use of "some" of the audio and video clips (M = 2.15, SD = 1.09, range = 1 
[none] to 4 [all]): Three of the students made use of all of the supporting audio and video clips, four accessed 
"most", six used "some", and eight used "none". The media access rate was somewhat lower for officials (M = 1.50, 
§D = 0.53), with four out of the nine using "some" of the media and the another five using "none." Four students 
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and two officials reported difficulty accessing the media, as a result of computer set-up problems (not enough 
memory, no audio capability, or software improperly installed). 

We asked students and officials for their perceptions of the twelve case ancillaries— supporting documents or 
media designed to flesh out and provide detail to the case. We wanted to know if the ancillaries were necessary to 
their understanding of the case. Responses were on a 4-point scale (1= not at all necessary, 2 = somewhat necessary, 
3 = helpful, 4 = very necessary) and are reported here with student and official ratings combined. 

The most useful ancillaries tended to be text-based and those most directly linked to instructional design 
practices (Meeting Notes with Workshop Content, Goals, Objectives, and Evaluation Plan, M = 3.59, SJ) = 0.63; 
Results of Formative Evaluation, M = 3. 50, ££) = 0.66). An exception here was the "Project Management Chart," 
which ranged just above "somewhat necessary" (M = 2.27, S]3 = 0.92). Ancillaries considered to be less useful 
included two of the media files: "Lucky Larry TV Spot" (video clip; M = 1-92, SJ) = 0.89) and "Mr. Tuthill's 
Address" (audio clip; M = 2. 16, SD = 0.85). 

Quality of the Case 

Fourteen of the students and six of the officials took time to comment on their positive feelings about the 
case. Six respondents commented on the realism of the case, with remarks such as "The variety of information 
seemed very reflective of the kind of data one would get in real life," and "I could 'see' this actually happening!" 

The depth and complexity of case events was generally thought to be effective for provoking student 
analysis and synthesis (eleven respondents addressed this positive quality). One student commented, "Working on a 
case provided a way to review [my] entire course of studies." Most officials likewise found the case and the analysis 
process worthwhile. An official commented that she was certain to have learned as much as the students, while 
another wrote: 

"The case study was successful in evoking a rich environment that included a number of possible courses of 
action. It provided a pretext for trying out theories and strategies, but just as importantly, noting where our 
theories came up short or fell completely silent." 

One official, however, noted that the case evidenced a "predominance of secondary information, i.e., 
description of people instead of encounters with them," while another reflected on the limitations of cases: "Trying 
to be so realistic, you end up being somewhat fake." 

A single respondent felt that the case provided "too much" ancillary material, while four others felt that the 
audio and video media were not necessary, as expressed in this comment: "Transcripts provided the information we 
needed." For one of the teams, Web access was primarily text-based, making audio and video access problematic. 

Consideration of the Case 

Our interviews suggest that teams actually employed a variety of approaches to organization and response 
creation. At the outset of the competition almost all of the teams, communicating via e-mail, negotiated schedules 
to set up meetings and issued requests that team members come to the first face to face meeting having read the case. 
One team went further: 

"....each of us on our own had addressed the questions and e-mailed them to each other. So there was an 
exchange of ideas before we sat down to discuss them." 

Most teams actually met between two and three times as a whole group. The initial meetings were used for several 
purposes: organizational minutiae (e.g. numbering pages of the printed out case to facilitate later discussion), 
divvying up tasks, and brainstorming ideas about the case. One team expressly used the initial meeting to take each 
other's measure. 

"We focused initially on the practice case study and addressed those questions and that was helpful to us to 
establish the group dynamics. Everybody kind of showed themselves during that time, so we knew what to 
expect" 



Two teams broke the case analysis task into "chunks" by "questions" (issues, perspectives, knowledge, 
actions, and consequences), meeting initially to divvy up the parts to team members according to their perceived 
strengths. "We discussed our strengths and decided, 'You know, I know more about this and I'd like to do this...'" 
Team members then went off and, working individually or in pairs, developed an answer to a particular part. These 
teams later came together to discuss these individual contributions or "negotiated responses via e-mail." 

In a third team, members composed individual answers to all of the questions, then came to meetings for 
discussion. One writer/editor composed the entire response based on these conversations. 

A fourth team used a very different strategy. These team members composed their entire response together, 
working at one computer: 

"We had three hours to really discuss the case. We took notes the whole time in sort of bulleted form. Then 
we came back in the second three hours and composed-distilled out of our notes what we wanted to say and how 
we wanted to say it. In my opinion we were very efficient in the process." 

Collaboration 

Collaboration was an important factor in teams' perceptions of their own effectiveness. Fifteen individuals 
remarked on this, making comments such as, "What worked was having to enter into collegial dialog, negotiating, 
arriving at consensus," "Working together with others who have different perspectives and information bases helped 
expand mine," and "We had some great discussion; you would have loved it!" 

The presence of conflict appeared to vary greatly among teams. In our interviews, two students indicated an 
absence of conflict on their teams, with one lamenting this fact: "I’d say that one problem was maybe that we were 
too similar-that may have restricted us." Other participants related that team conflicts, both potential and actual, 
seemed to evolve from differences in background and experience, educational training ("we weren’t in a common 
frame of reference of what we were studying"), and writing styles, in addition to miscommunications. 

Methods for dealing with differences in opinion ranged from ignoring outliers to incorporating ideas into the 
whole response in a compromise: "On issues where we could not come to closure, generally we included the input 
of both people." Most notable, however, was the enthusiasm expressed by some of the participants for the rough 
and tumble nature of discussions in which differences were ironed out: 

"You know, everybody needs to go through that. That’s so essential. What was neat about it was that we were 
quite a blend of personalities. You know, we all learned something from one another in this whole process and 
that's what it should be about." 

"Because of the conflicting viewpoint, to bring the group to consensus we all had to have a good understanding 
of what was going on and that required getting deeply into the case. Two or three of the people said that they 
really liked this approach to working on it and getting the benefit of other people’s ideas." 

"And what ensued was good. I didn’t have all the right answers. A lot of things I would have designed might 
have come undone had it not been for teammates. There are many things that they put in that I hadn’t thought 
of. I don’t care who wins this thing. I don't. But, I tell you straight out, I feel like I’m a winner already 
simply because I learned so much from it." 

Competition 

We wanted to hear from our respondents on another important aspect of the case competition— the 
competition itself. They had been involved in a case event in which a winning response would be identified. How 
did that influence the team members, both in the quality of their participation and in the crafting of their response? 

Students expressed positive attitudes towards competition, with many comments about its motivating 

effect: 

"I think that (the competition) was crucial to keeping everybody engaged. If there had been no competition, it 
would not have been a vicarious experience of relatively deep engagement with Dundee High School." 
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"I think we had a team spirit, or a university spirit. We knew that there were other schools and that possibly 
they were coming from different theoretical perspectives or different influences of different professors and they 
may take a different approach. It aroused our curiosity." 

"Whether I like it or not, I think competition serves a purpose. We want to try to make things as cooperative 
as possible, but competition produces a different edge and that can be good when it’s properly channeled. I think 
it's good for students to learn that the world involves competition." 

When we asked how the respondents felt about being judged with one team’s response being declared a "winner," we 
received some interesting observations, 

"The judging may have influenced us in the beginning. But... we got lost in it. I think the competition just 
sort of took a back seat." 

"We are more interested as a group in seeing what other people have said. We don’t really care how the judges 
say we did." 

"The fact that it was judged added immeasurably to its attractiveness as a competition case for me. In fact I’m 
not sure that I would have participated had it not been judged. I don’t think I would have." 

Including one negative comment: 

"A competition means that somebody, the winner, does the thing the best. So let’s say the objective is to 
learn. Let’s say you learn, but you lose. I know that when I lose, I feel like I didn’t learn." 

An event official encouraged consideration of the benefits competition and collaboration each provide, and asked, 
"How can the rules be adjusted to allow the best of both worlds?" 

Case Responses & Event Outcomes 

The three provocateurs developed specific questions for individual teams and a common question to be 
answered by all teams. The specific questions included: 

"There appeared to be tacit approval by all members of the committee and the community that a series of 
workshops was the most effective way of getting high school students to become empathetic, effective problem 
solvers. Do you agree? If so, justify large group workshops as the most effective approach. If not, describe 
instruction/leaming experiences that may be more effective in accomplishing the objectives set forth by the 
Workplace Readiness Committee." 

and 

"Please compare your own action plan against that proposed by the Workplace Readiness Project Committee. 
Will it fit within the constraints of the project (i.e., a small grant for a series of workshops)? How does your 
plan better address the target population? Is it grounded in the context of practical activity?" 

Meanwhile, all teams were posed the following common question: 

"It appears that one of Terry’s major failings, as with so many instructional designers, was in not conducting 
any sort of context analysis to describe the organizational, socio-cultural context in which this process was to be 
played out. How should she have done this? What do you believe that she would have found? How would that 
have affected the design of the instructional/ learning activities that were used to engage the students?" 

The five judges reviewed the team's (blind) case responses and responses to provocateur questions over a 
two- week time period. At the end of this time, they returned written- comments for each team and the completed 
response rating form. 




Team ratings on the evaluation items (1 = low, 4 = high) were averaged across the eight items and five 
judges. These average team ratings ranged from a low of 2.79 (SD = 0.49) to a high of 3.3 (SD = 0.59), suggesting 
that all the teams did fairly well. 

In general, the judges felt positively about the teams' overall performance (M = 3,10, SD = 0.40), and their 
ability to identify the important issues (M = 3.08, SD = 0.64), define the perspectives of key players (M = 2.96, SD 
= 0.77), apply professional knowledge (M = 3.08, SD = 0.63), specify future action (M = 3.02, SD = 0.70), and 
anticipate the consequences of the action (M = 2.82, SD = 0.51). They also felt that teams' responses to the 
provocateur questions (common question M = 2.80, SD = 0.58; specific question M = 2.96, SD = 0.41) 
demonstrated some insight into professional practice. 

When we spoke with event officials, we pursued the relationship between teams' case responses and 
perceptions of the teams' design expertise. While definitive relationships were not found here (some officials felt the 
case responses were strong and others less so), some valuable insights were offered. Two officials reflected on the 
relationship between ID theories and training and the case responses they reviewed: 

"The responses were kind of light weight... They were trying too hard to show what they had learned, you 
know, glib stuff that you learn in a master's program in instructional design, without too much integration to 
the realities of the case." 

"The case brought out the inadequacy of some of our theories... Even if you try to apply all of that knowledge 
there's still so much more you need to know in order to succeed. Are these things being taught in our classes? 
Maybe or maybe not. It's a stark assessment of our theories as we look at these rich cases. We would have to 
conclude that we are only partially giving students the tools that they need." 

However, one official noted the difficulty in making assumptions about design expertise when teams had merely 
responded to the case and not developed an instructional design: 

"I could get a sense whether they had concepts like needs assessment, evaluation, or context analysis but I 
couldn't really get a sense that they could design a program of instruction." 

Design & Management of the Case Event 

Several issues related to the design and management of the event emerged as important. A limit on time 
allowed for team meetings was seen as difficult by one student: "It takes a great deal more time than [six hours] to 
put a team together so that they function as a team." The limit on length of case response (1,250 words) was viewed 
as problematic by another: "Answers such as those we want from case-based learning cannot and should not be 
relegated to lists, cookbook-like two sentence answers, or sound bites." However, the time and length limits were 
seen as positives by five other student respondents: "At first, I didn't like the word limitation or the strict time 
limits, but I think it's in our best interest." "We had to be succinct and to the point." "Setting time limits was a 
stroke of genius." 

Noting the two stages of team response (case response, response to provocateur questions), a student added 
that "two levels of group input is far better than a one-time effort." The use of provocateur questions, while seen as 
a valuable concept, was not satisfying to three of the students: "The questions were not very challenging and did not 
provide an opportunity for additional analysis." "They looked like they had been written before our response." Hie 
need for better development of the provocateur's role was noted by two of the provocateurs: "I would have preferred a 
greater degree of interaction with my peers." "Time constraints were tough, but I wish I had been more proactive in 
discussing our questions with the other provocateurs. We could have been more instructive in our questions to the 
teams." 

Several students and officials reported a desire for more sharing of case responses and discussions between 
sites. The top two case responses were posted to the Web site after judging was completed, but at least one student 
and one official wanted to read all of the case responses. The official commented that even though he hadn't been on 
a team, that "I felt an urge to talk it over, wishing I could argue the key points of the case and my particular 
solutions. I wonder if student teams also felt a desire to debrief further." 

The time within the semester for this ease event was seen as a problem (students were involved in event 
activities during the month of April). Nine students commented on this, and eight of them suggested that the case 




event occur earlier in the semester while one of them (on a quarter system) indicated that it should come later. 
Coordination of submissions and communications with the participants was viewed as effective, with respondents 
remarking on the enthusiasm of the event staff, and the quick turn-around for submissions. 

Value of Participation 

We asked students and officials two questions eliciting their opinions about the value of case methods and 
this case event (Likert response options were from 1 [strongly disagree] to 4 [strongly agree]). Students felt that the 
case study method is valuable for developing ID expertise (M = 3.81, SD = 0.40), a perception that was shared by 
the officials (M = 3.56, SD = 0.53). They also expressed enthusiasm for the value of this case event in preparing 
students for future ID projects (students M = 3.62, SD = 0.50; officials M = 3.67, SD = 0.50). 

Most of the officials were positive, with several making enthusiastic claims: 

"...probably the single instructional strategy innovation that could make the biggest difference in education," 
and 

"In my mind the case competition format is a watershed event in the history of teaching instructional design... 
The case competition format allows students to really dig into a scenario and apply what they have learned about 
the instructional design process." 

While another official expressed more skepticism about cases in general, arguing that cases should not substitute for 
real design experiences: 

"Trying to represent reality, when reality is already there, many not be the best use of our energies." 

Discussion 

According to our follow-up survey (response received from at least 65% of students and 75% of officials), 
the ID case competition was a valuable experience for those involved. In expressing their reasons for participating in 
the case event, students and sponsors alike noted the potential for learning about instructional design. 

Students demonstrated enthusiasm in their participation. Prior to the event, all participants reviewed the 
Web site and a practice case, and nearly two-thirds of the students discussed the practice case. While cases had been 
used previously by somewhat less than half of the students and three-fourths of the officials, few reported specific 
experience with ID cases. Collaboration was an important factor in teams’ perceptions of their own effectiveness. 
Students commented on the value of collegial dialog, negotiation, and consensus decisions. Many students noted the 
motivating aspects of the competition in their responses, commenting on their pride at representing their schools and 
the edge, or focus, that the competition brought to the case analysis experience. Similarly, the prospect of being 
judged was viewed positively by most of the students. 

Most of the participants felt that the "Trials of Terry Kirkland" case was realistic— detailed, complex, and 
providing a number of courses of action, while several noted the deficiencies of the case as compared to real design 
experience. The Web proved to be a useful medium for distributing the case materials, with most participants 
reporting that they reviewed the case both on-line and via printouts. The most useful ancillaries tended to be text- 
based and those most directly linked to instructional design practices. Because we provided transcripts for all of the 
audio and video clips, we were not surprised that usage of these media was so low. We considered these materials to 
be supporting in nature— materials included to round out and add realism to the case. Because we suspected that media 
access would be problematic (and for some participants this was the case), we provided transcripts for these materials, 
making the audio and video clips even less necessary. 

Teams employed a number of different approaches to their case analysis and response development process. 
Most teams met two or three times, with the initial meeting being used to develop action plans and to do some 
preliminary brainstorming. Some teams divided up the case response, while in other teams all members answered all 
questions, either separately (coming together later to compare responses and develop the final response) or together 
(developing each portion of the case response within a meeting). There appeared to be little conflict, and when there 
was conflict it was resolved productively, with several participants noting that the conflict that did occur was a 
valuable part of the case analysis process. 
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While limits on team discussion time and response length were seen as restrictive by a few students, a 
number of others indicated their support for these guidelines, noting that they knew these limits were in their best 
interests and that the limits encouraged focused meeting time and succinct responses. The implementation of the 
provocateur role left something to be desired for some of the students and provocateurs--students noted lack of 
specificity and challenge in the provocateur questions, while provocateurs wished for more discussion opportunity 
with their provocateur peers. 

Judges felt that teams performed well in their case responses; all but two of the judges' thirty team ratings 
(five judges x six teams) on the overall performance item indicated agreement or strong agreement that team 
performance was excellent. Ratings on each of the specific criteria (identification of issues, application of 
professional knowledge, etc.) indicated general agreement that team performance was appropriate/effective. 

In their consideration of team case responses, several officials noted what they felt to be evidence of 
limitations in our ID models and student preparation, while the comments made by another suggested that it was 
difficult to make assumptions about design expertise since the teams had not been required to develop an 
instructional design in response to the case. 

Nonetheless, all of the responding participants (students and officials alike) agreed or strongly agreed that 
case methods are valuable for developing ID expertise, and that participation in this case competition was valuable in 
preparing students for future instructional design projects. 

Limitations of these findings 

It is possible that participants not responding to our survey had different feelings about their participation 
and about the value of case methods for instructional design. Participants were contacted about the survey three 
times and about the interview twice, so we feel that all who wished to contribute their perceptions were given the 
opportunity. 

We know that participants felt positively about the value of this case-based experience, but we cannot know 
whether participation will actually improve students' future instructional design efforts, teamwork, or consulting 
skills. Inquiries considering the relationship of case methods to these longer-term outcomes will be important. 

Future research and development directions 

"Do it again!" We will host a second ID case event during the spring of 1997. We plan to build upon the 
successes of this first attempt and to capitalize on the excellent suggestions made by first-year participants. We will 
expand on opportunities for collaboration both within and across sites, while still offering teams an opportunity for 
healthy competition. We will also charge teams with developing an instructional design or a needs assessment in 
response to each case. Three cases will be offered: a Practice Case, a Discussion Case, and a Competition Case. 

The Practice Case (we will use "The Trials of Terry Kirkland" for this purpose) will be available for any 
type of use at any time. In addition to the case materials, winning case responses and judges' comments, we have 
included a teaching note which includes a variety of questions to encourage case exploration and discussion. We have 
also added the perspectives of three experts on the case, so that students can consider some different points of view 
after developing their own response to the case. 

The Discussion Case will be provided in a similar Web format. Participating students will discuss the case 
with faculty and other students at their institution as they develop their case response in the form of an instructional 
design. Three provocateurs will each assume the perspective of one of the principals in the case, and will read team 
responses and pose questions for teams to respond to from these perspectives. The case and question responses from 
all teams will be posted to the web site. Finally, discussion of the case and responses will be encouraged across 
sites, through an electronic mailing list. 

The Competition Case will also made available in Web format. Teams will be asked to develop an 
instructional design or a needs ass(£§Ynent in response to this case. We will be working to include the Provocateurs 
and Judges more integrally in the case activities and discussion, and the Judges will provide written evaluation of 
case responses. The timing of this event will be moved to an earlier point within the semester, to make student 
participation easier. 

Use of audio and video media. 

We plan to continue experimenting with provision of multimedia-based materials in cases; multimedia has 
the potential to provide encounters with people instead of descriptions of them (as suggested by an official). As we 
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do so, we will need to consider how to make these materials available to the widest possible audience. While 
transcripts provide the verbal contents of an interaction, they may not be able to provide a sense of underlying 
emotion or political charge that can be just as important. 

We hope that through activities such as those described above, we will be able to define techniques for 
providing valuable learning experiences in instructional design. 
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Over the past two decades, a number of researchers have examined the effect of cooperative learning on 
student achievement and motivation. Reviews of research have generally suggested that cooperative learning has a 
positive influence on student achievement, productivity, transfer of learning, time on task, and attitude (Johnson & 
Johnson, 1989; Sharan, 1980; Slavin, 1990). According to Widaman and Kagan (1987), the results of these reviews 
have led many educators to erroneously conclude that cooperative learning is superior to traditional instruction for all 
students because it results in greater achievement by average students. However, even advocates of cooperative 
learning have indicated that some learners are more predisposed than others to act cooperatively (Johnson and 
Johnson, 1989) and that this predisposition may influence how students cooperate when they work with others 
(Slavin, 1983). 

There is some empirical evidence that a student’s need for affiliation may influence outcomes in a 
cooperative learning setting. The need for affiliation is represented by a desire to participate in cooperative, 
noncompetitive activities and by a desire for close, friendly relationships with others (McClelland, 1965, 1976). 
Individuals with a high need for affiliation are more friendly, sociable, and cooperative than those with a low need for 
affiliation (Jackson, 1974). A recent study by Klein and Pridemore (1992) revealed that need for affiliation interacted 
with cooperative/individual learning to influence performance and attitude. College students with high affiliation 
need who worked alone performed worse than students in all other conditions when asked to apply what they had 
learned. Furthermore, Chan (1980-81) found that high school students with high need for affiliation reported better 
attitudes toward cooperative learning than those with low need for affiliation after using both cooperative and 
individual instruction. 

A few other researchers have reported that social orientation may influence how students perform in 
cooperative learning settings. Hall et al. (1988) found that pairs of college students with moderate to high levels of 
social orientation outperformed pairs with low levels of social orientation; students with a low social orientation 
performed better when working alone. Widaman and Kagan (1987) reported that cooperatively-oriented elementary 
school students performed better when placed in cooperative learning structures, while competitively-oriented 
students performed better in competitive learning structures. Sutter and Reid (1969) found that college students with 
high levels of sociability performed better than introverted students on cooperative computer-assisted instruction, 
while introverted students performed better on individual CAI. Finally, Jones (1995) found that pairs of college 
students with a high preference for group work spent more time working through computer instruction, exhibited 
more cooperative behaviors, and more off-task behaviors than pairs with a low preference for group work. 

The purpose of the current study was to examine the effect of cooperative versus individual learning 
strategies and the need for affiliation on achievement, attitude, and student interactions. The study is a continuation 
of a program of research on how cooperative learning can be effectively implemented with instructional television. 
Johnson & Johnson (1994) have suggested that informal cooperative learning structures such as paired discussions 
can ensure that students are cognitively active during the presentation of a videotape. Furthermore, Adams, Carson, 
and Hamm (1990) have speculated that cooperative strategies will enhance learning and motivation when 
instructional television is presented to students. However, few research studies have been conducted to examine the 
effect of using cooperative learning strategies with instructional television. 

In this study, subjects classified as high or low affiliation used either a cooperative or individual learning 
strategy while receiving instruction from a television lesson. Based on previous research, it was hypothesized that 
instructional condition would interact with need for achievement to influence learning and attitude. It was expected 
that high affiliation students would achieve more and report better attitudes under the cooperative condition and that 
low affiliation students would achieve more and report better attitudes under the individual condition. Furthermore, it 
was hypothesized that high affiliation dyads would exhibit more cooperative interactions and more off-task 
interactions than low affiliation dyads. 
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Design and Subjects 

A 2 X 2 factorial design was used in this study, with instructional method (individual versus cooperative) 
and need for affiliation (high versus low) as the independent variables. The dependent variables were achievement and 
attitude. Data for student interaction behaviors were also collected for subjects in the cooperative treatment. 

Subjects were 126 undergraduate education majors (34 males, 92 females) enrolled in the first semester of a 
teacher training program at a large southwestern university. All subjects were enrolled in a required course in 
educational psychology. Although students in this course were required to participate in one research study during 
the semester, participation in this particular study was voluntary. 

Materials 

Materials used in this study were an instructional television lesson and an instrument to measure the need 
for affiliation. The instructional television lesson was from the series Instructional The ory: A nine unit mini-course 
(Gerlach, 1973). The lesson included a videotape and a workbook that provided instruction on the topic of 
objectives-based assessment. The videotape was divided into seven segments which presented information and 
examples on the content of the lesson. The videotape portion of the lesson was approximately 30 minutes in 
length. After each segment, the videotape instructed subjects to turn to their workbook for practice and feedback on 
the content presented in that segment. For example, Segment 4 provided instruction on the use of paper-and-pencil 
tests, interviews, and observations of student performance or product. After providing information and examples of 
these three types of objectives-based assessment, the tape presented viewers with three instructors who wished to 
evaluate a student's work of sculpture. The videotape directed subjects to "Turn to Exercise 4 in your workbook" 
where they were asked to "Describe the best type of objectives-based assessment for this situation." The workbook 
then provided written feedback to this practice item on the following page. 

The affiliation scale of the Personality Research Form-E was used to measure need for affiliation. This 
scale consists of 16 items that measure the degree to which an individual is motivated to affiliate with others. A 
true-false format is used to indicate whether or not a person agrees with statements such as "Sometimes I have to 
make a real effort to be social" and "I spend lots of time visiting friends." According to Jackson (1974), a high score 
on this scale suggests that the individual enjoys being with other people, accepts people readily, and makes an effort 
to have friends and maintain associations with others. Norming data indicate that the mean for this scale is 8.6 (SD 
= 3.35) and that the internal consistency reliability is .86 when used with college students (Jackson, 1974). For 
subjects in the current study the mean was 9.98 (SD = 3.59), and the range was 2 - 16. 

Procedures 

Several weeks before the treatments were implemented, all subjects completed the need for affiliation scale. 
A median split was used to block subjects into high and low categories of affiliation. Subjects with scores at or 
above the median (Md = 10) were assigned to the high affiliation category (M = 12.4, SD = 2.0, n = 75) and those 
with scores below the median were assigned to the low affiliation category (M = 6.4, SD = 2.1, n = 51). Subjects 
blocked by affiliation score were randomly assigned to one of the two treatment conditions (cooperative or 
individual). Cooperative dyads were then formed by randomly pairing subjects from the same affiliation category 
(i.e., a high affiliation subjects were paired and a low affiliation subjects were paired). 

During the study, each treatment condition was implemented in separate rooms; each room had more than 
one individual or dyad present at a time. Subjects in both treatment conditions were informed that they would be 
viewing an instructional television program on objectives-based assessment and that they would be using a 
workbook to receive practice and feedback on the content of the lesson. Subjects were told to write the answer to 
each practice exercise in the workbook and read the feedback that followed each exercise. Additionally, all subjects 
were informed that a short test would follow the lesson. 

Subjects received specific directions for implementing individual versus cooperative strategies. Subjects 
working alone were each given a workbook, instructed to work independently during the lesson, and told to do their 
best work. Individuals were also informed that they would receive bonus points toward their course grade if they 
achieved 90% or better on the lesson test. Each cooperative dyad was given a workbook and told to work together 
during the lesson, discuss all practice exercises and any disagreements over the answers, and discuss the given 
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feedback. Cooperative subjects were also informed that they would individually complete the lesson test and would 
receive bonus points if both partners achieved 90% or better on this test. 

After the above instructions were provided, the videotape was started for each treatment condition. When 
Segment 1 was completed, the tape was stopped and subjects completed Exercise 1. When subjects indicated that 
they were ready, the videotape was started again. This cycle was continued until all seven sections of the lesson were 
completed. 

During the lesson, observers watched the dyads work and recorded instances of student interactions. An 
observer also watched individuals work and took notes on their behaviors. Upon completion of the lesson, all 
workbooks were collected and each subject individually completed an attitude survey. Each subject then took the 
posttest. 

Criterion Measures 

Criterion measures used in this study were student achievement and attitude. In addition, student interaction 
behaviors for subjects in the cooperative treatment were observed and recorded. 

Achievement was measured using a 15-item, constructed response posttest. The items were developed to 
evaluate student mastery of the instructional objectives for the lesson on objectives-based assessment. The posttest 
measured both application and knowledge of the lesson content. The application portion of the test consisted of ten 
items and the knowledge portion consisted of five items. Each section of the test was worth a total maximum score 
of ten points. Individual answers were checked against a scoring key and points were assigned for each answer. 
Partial credit was given for questions that required a multiple response such as "List three types of objectives-based 
assessment." The internal-consistency reliability of this posttest was .86 (Klein, Erchul, & Pridemore, 1994). 

Attitude was assessed using a 10-item, paper-and pencil survey. This survey consisted of six questions 
from the Instructional Materials Motivation Scale (Keller, 1986) that measured student satisfaction toward 
instructional activities and four items that assessed the degree to which a student would be willing to return to tasks 

like those used in the study. A five-point Likert scale (I = not true, 5 = very true) was used to respond to the 

following items: 

(1) Participating in the activity gave me a satisfying feeling of accomplishment; 

(2) The practice and feedback helped me feel satisfied while participating in the activity; 

(3) I enjoyed the activity so much that I would like to participate in a similar activity; 

(4) I really did not enjoy the activity; 

(5) It felt good to successfully complete this activity; 

(6) It was a pleasure to work on such a well-designed activity; 

(7) I would like to receive more instructional television lessons in the future; 

(8) I would like to learn more about objectives-based assessment in the future; 

(9) I would like to participate in future activities that allow me to work with another person; 

(10) I would like to participate in future activities that allow me to work by myself. 

The Cronbach alpha internal-consistency reliability estimate of this survey was .83. 

The number of student interactions exhibited by cooperative dyads were observed and recorded on an 
observation sheet. This observation sheet included interaction behaviors that other researchers have suggested as 
necessary for successful group work (Klein & Doran, 1997; Klein & Pridemore, 1994; Webb, 1982, 1987). These 
interaction behaviors were grouped into the four categories of helping behaviors (asking questions, answering 
questions, giving unsolicited hints, suggestions, or explanations) on-task group behavior (taking turns, sharing 
materials, group discussion of content), on-task individual behavior (assuming control, taking notes, working 
alone), and off-task behavior (talking to other about something unrelated to the lesson and non-verbal actions such as 
reading a newspaper). 

A trained observer was stationed among five dyads to observe each dyad for two minute intervals during the 
lesson. The observer placed a mark on the observation sheet when a dyad exhibited an interaction behavior. Prior to 
the study, observers watched a videotape of one dyad working through a lesson and used the data collection form to 
record interaction behaviors. Reliability of observations was based on observers having similar totals for each set of 
behaviors exhibited by this videotaped dyad. The inter-rater reliability between observers was .90 for helping 
behaviors, .75 for on-task group behaviors, .90 for on-task individual behaviors, and 1.0 for off-task behaviors. 
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Data Analysis 

Data for 122 subjects were included in the analyses, since inspection of residuals indicated that posttest 
scores for four subjects were extreme outliers (z> 2.25). Multivariate analysis of variance (MANOVA) was used to 
test for an overall difference between groups on the posttest. This analysis was followed by univariate analyses on 
the knowledge and application portions of the test. MANOVA was also used to test for an overall difference between 
groups on the attitude survey. This analysis was followed by univariate analyses on the individual attitude items. 
The number of interaction behaviors exhibited by subjects in the cooperative treatment were totaled for 26 dyads and 
separate chi-square analyses were conducted on each category of interaction behavior. Interaction data for five high 
affiliation dyads were randomly eliminated prior to conducting these analyses in order to create a balanced design (13 
high and 13 low affiliation dyads). Alpha was set at .05 for all statistical tests. 

Results 

Achievement 

Achievement was measured using the 15-item, constructed response posttest. The posttest measured both 
knowledge and application of the lesson content. Mean scores and standard deviations for both portions of the 
posttest can be found in Table 1. 



Table 1 . Mean Scores and Standard Deviations for Achievement 



Type of Items 
Condition 




Knowledge 


Application 


Total 


Individual Learning 


Low Affiliation 


M 


5.42 


7.08 


12.17 


(n = 24) 


SD 


2.19 


1.70 


2.77 


High Affiliation 


M 


5.67 


5.60 


12.33 


(n = 36) 


SD 


1.76 


1.15 


2.13 


Total 


M 


5.57 


6.26 


12.26 


(n = 60) 


SD 


1.93 


1.58 


2.39 


Cooperative Learning 


Low Affiliation 


M 


4.54 


6.83 


12.90 


(n = 26) 


SD 


2.06 


1.52 


3.41 


High Affiliation 


M 


5.00 


6.81 


12.96 


(n = 36) 


SD 


2.11 


1.03 


3.34 


Total 


M 


4.81 


6.82 


12.93 


(n = 62) 


SD 


2.09 


1.29 


3.32 



A MANOVA conducted on the posttest data did not reveal a significant multivariate effect for either 
instructional method or need for affiliation. However, univariate analysis conducted on each section of the posttest 
revealed that instructional method had a significant effect on knowledge acquisition F(l, 118) = 4.32, p < .05. 
Subjects who worked alone (M = 5.57, SD = 1.93). performed significantly better on the knowledge portion of the 
posttest than those who worked cooperatively (M = 4.81, SD = 2.09). No other results were found for achievement. 
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Attitude 

Attitude was measured using the 10-item survey. MANOVA revealed a significant main effect for 
instructional method F(10, 109) = 2.26, p < .05. Univariate analyses indicated that subjects who worked alone were 
significantly more likely than those who worked cooperatively to agree with the statement: I would like to 
participate in future activities that allow me to work by myself, E(l» 1 18) = 4.59, p < .05. 

MANOVA also revealed a significant main effect for affiliation, F(10, 109) = 2.29, p < .05. High 
affiliation subjects were significantly more likely than low affiliation subjects to agree with the statement: I would 
like to participate in future activities that allow me to work with another person, F(l, 118) = 7.34, p < .05. 
Furthermore, low affiliation subjects were more likely to agree with the statement: I would like to participate in 
future activities that allow me to work by myself, E(l. 118) = 3.67, p = .058. 

Student Interactions 

High affiliation dyads exhibited 90 helping behaviors, 97 on-task group behaviors, 47 on-task individual 
behaviors, and 25 off-task behaviors. Low affiliation dyads exhibited 92 helping behaviors, 67 on-task group 
behaviors, 57 on-task individual behaviors, and 8 off-task behaviors. Chi-square analyses were performed on each of 
the four categories of student interactions to determine the influence of affiliation. These analyses indicated a 
significant difference between high and low affiliation dyads on two of the four behaviors. Results revealed that high 
affiliation dyads exhibited significantly more on-task group behaviors (discussion of content, sharing materials, 

taking turns) than low affiliation dyads, ^ = 5.49, p < .05. In addition, high affiliation dyads exhibited significantly 
more off-task behaviors than low affiliation dyads, 2 = 15.13, p < .05. 

Discussion 

The purpose of this study was to examine the effect of cooperative versus individual learning and the need 
for affiliation on achievement, attitude, and student interactions. College students classified as high or low 
affiliation used either a cooperative or individual learning strategy while receiving instruction from a television 
lesson. Results did not reveal a significant difference between subjects who used cooperative and individual learning 
strategies when overall achievement and application were tested. This is likely due to the instructional materials 
used in the study. The television lesson was designed following a competency-based approach and included 
objectives, information, examples, practice, feedback, and review. Some researchers have suggested that studies 
comparing individual and cooperative learning strategies do not consistently favor small group methods when well 
designed instructional materials are used (Bossert, 1988-89; Cavalier, 1996; Klein & Doran, 1997; Snyder, 1993). 
Others have indicated that many studies which produce positive results in favor of cooperative learning have 
compared carefully designed materials for groups to poorly designed instructional materials for individuals (Bossert, 
1988-89). In the current study, subjects in both the individual and cooperative treatments used well designed 
instructional materials. 

While overall achievement and application were not influenced in this study, results indicated that subjects 
who worked alone performed significantly better on the knowledge portion of the posttest than those who worked 
cooperatively. Furthermore, subjects who worked alone expressed better attitudes than those who worked 
cooperatively toward future activities that require individual work. These findings partially support the work of 
others who have suggested that individual strategies may be more effective than cooperative strategies for enhancing 
learning and motivation during instructional television lessons (Klein, Erchul, & Pridemore, 1994). 

Results of the current study also revealed that need for affiliation was related to attitude toward future 
learning activities. It is not surprising that high affiliation students expressed a positive attitude toward future group 
work and that low affiliation subjects expressed a positive attitude toward future individual activities. Other 
researchers have reported that need for affiliation is related to attitudes when student implement individual and 
cooperative learning methods (Chan, 1980-81; Klein & Pridemore, 1992). 

In addition to attitudes, results indicated that need for affiliation was related to student interaction behaviors. 
High affiliation dyads exhibited significantly more on-task group behaviors (discussion of content, sharing materials, 
taking turns) and significantly more off-task behaviors than low affiliation dyads. These results support the notion 
that individuals with a high need for affiliation are more cooperative and social than those with a low need for 
affiliation (Jackson, 1974; McClelland, 1976). The findings are also consistent with results reported by Jones 
(1995) who found that pairs of college students with a high preference for group work exhibited more cooperative 
behaviors and more off-task behaviors than pairs with a low preference for group work. 

erJc 




Combined with previous research, the current study provides some implications for educators who plan to 
implement cooperative learning strategies with media that was originally designed for individual learning. Results 
suggests that educators should consider student characteristics such as need for affiliation when forming cooperative 
learning groups. Furthermore, while some theorists have suggested that cooperative learning structures can enhance 
active learning during videotaped instruction (Adams, Carson, & Hamm, 1990; Johnson & Johnson, 1994), the 
current study indicates that cooperative learning does not always increase learning from instructional television that 
has been systematically designed. Future research should continue to examine the effect of learning structures and 
student characteristics when students participate in mediated lessons. 
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Abstract 

Part of what influences educational practice is the constant visualization of gender stereotypes throughout 
our society in various forms, ranging from the older technologies to the new technologies. Further, the imagery of 
computer technology itself as male turf has now been carried into the WWW through graphic advertisements. Male 
administrators continue to make decisions about school practice and this influences the implementation of the new 
distance education technologies. We must take stock of the pervasive messages of gender stereotypes, their 
tremendous influence on children and adults, and be aware of how these stereotypes can bring biased value systems 
into what seems to be otherwise technologically innovative environments. 

Historically, society has carried messages to the public that are laden with gender stereotypes. These 
messages reach people through many forms of communication, including the spoken word, print-based and electronic 
text, still images, full-motion images, auditory channels, and various combinations of these forms. The stereotypes 
permeate our magazines and newspapers, books, television programming, entertainment resources, advertisements, 
educational materials, school practices, work environments, and interpersonal contacts. They affect our child-rearing 
practices, choices of products and activities, fashion design, attitudes, value systems, aspirations, self concepts, 
opportunities, access to information, social contacts, and wage-earning potential. 

The stereotypes are so deeply ingrained into our society that even when people recognize the 
discriminations, they accept it as the way things are. Thus parents unwittingly go along with the toy and garment 
manufacturers, buying passive toys and pastel-colored clothing for their daughters, and action or problem-solving 
toys and brightly-colored clothing for their sons. Girls are groomed for passive and supportive roles while boys are 
groomed for careers and leadership. Girls learn to dream of marriage, and boys learn to dream of independence. Girls 
learn to babysit while boys learn to play the stock market. 

Somehow these practices get passed along into the educational system and opportunities for school and 
work are influenced by gender. Part of what influences educational practice is the constant visualization of gender 
stereotypes throughout our society in various forms, ranging from the older technologies to the new technologies. 
Further, the imagery of computer technology itself as male turf has now been carried into the WWW through graphic 
advertisements. The WWW advertisments have the potential to reach an even larger audience and be more influential 
than the other forms of media which reach a more limited target audience. We must take stock of the pervasive 
messages of gender stereotypes and their tremendous influence on children and adults. First, let’s look at some of the 
ways in which people receive these messages. 

Developing Male and Female Stereotypes 

Girls and boys seem to mix well until they reach preschool age, when there begins a distinct emphasis on 
activities and treatment of children by gender. By the time they are five years old, there is a distinct value system at 
work within them that clearly specifies values that they pick up from messages within our culture, its fairy tales, 
media imagery, and so on. The effect of the value system can be seen in the toys that the children ask for, their 
hobbies, and the gender separation that evolves into things like birthday parties that are segregated by gender at the 
tender age of six. While this is not true for all girls and all boys, it is a pattern that is evident in our schools, in our 
homes, and in our society. If you doubt for one moment that it is true, then watch patterns of socialization as 
students walk to and from school or ask yourself if you have ever heard of the comment, “he plays like a girl” or 
“he runs like a girl” in reference to any sports activity. This familiar comment certainly is not flattering to anyone 
and does not recognize the achievement of women athletes, but instead seems to reflect the everlasting attitude of 
sports being a male domain. 

People believe what they are brought up with and what they see in the media. Thus cultural background 
influences beliefs and behavior, and so does the media. Take for example, the value systems passed on from a parent 
to a child, a teacher to a student, a religious leader to a congregation, or the media to a consumer. Now think about 
art and the way women have been portrayed through hundreds of years. Art forms have traditionally portrayed 
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women as beautiful, sexy, and passive. Think about the great painters of this world and they way they have depicted 
women as opposed to men. The influence of the imagery most certainly has had an affect on both men and women. 

Increasing attention to gender equity should yield more balance, especially in instructional materials, 
practices, advertising, and entertainment. Yet a close examination of gender as portrayed with the popular media, 
reveals a clear, consistent, and pervasive relationship that has deep historical roots, and that winds throughout our 
daily lives and perpetuates itself through its interweaving with society. Further, the role of girls and women in 
relation to the new media, has made little progress beyond that depicted with the now-traditional form. 

These biases continue to support instructional design in its historical sense by constantly feeding the old 
system while all but strangling attempts to pay serious attention to gender equity (McCormick, 1994; Gomick & 
Moran, 1972). Materials developed for use in public, private, and military schools as well as instructional messages 
delivered to the public through advertising, television, and public service messages continue to portray women and 
men in stereotypical ways. Despite attempts to correct this situation over many years, a recent study of computer 
clip-art images reveals that the stereotypes have invaded the desktop computing environment, with images of men 
depicted in leadership and authority roles, while women are depicted in subordinate roles (Binns & Branch, 1995). 

The complexities of this relationship are enormous, yet can be difficult to recognize, reveal, analyze, 
explain, and redirect. Like society itself the complexities reflect the dynamics of different situations in different 
ways, among different individuals. An examination of the complex relationship between instructional design and 
gender reveals inequities that result from a persistent pattern of practice. Recognizing the result of those inequities 
can be easier than finding the causes and correcting the problem at its root. 

Inequities that result from the practice of instructional design often go unrecognized because they emerge 
not just as a result of what has been done, but also as a result of what has been left undone. The neglect and 
omission of the female population reveal themselves in subtle ways on an individual basis, but as a collective result 
appear throughout society as something that begins in the home, and perpetuates itself through schooling and 
employment practices. If that was not the case, then there would be no need for recent efforts to attract girls into the 
study of math and science (Kable & Meece, 1994) and the number of distressing stories about females succeeding 
despite the myriad of obstacles (Aisenberg & Harrington, 1988; Clark & Corcoran, 1986; Frenkel, 1990; Gomick, 
1990) would no longer be told. 

The group of papers in this section present information about the way that gender is influenced by society, 
role expectations, advertisements, and educational decision makers. Images in advertisements that began in a print- 
based format, but have evolved to full-motion television, and now to the WWW continue to portray men and women 
in stereotyped roles. Males continue to dominate the computer culture, cyberspace, clip art, and the advertising 
about using technology professionally and productively. Male administrators continue to hold top administrative 
jobs in the professions, including the educational field, and their decisions most definitely influence the use of 
technology in classrooms, whether or not the teachers who are in those classrooms are consulted. This paper and the 
papers that follow address the obvious gender stereotypes in our culture and group patterns, print materials, 
television programming and commercials, cyberspace, and educational environments. 

Culture and Groups 

Images of our society reflect the stereotypes and the realities of gender tracking, often separating males from 
females. For example, one might think about a common media image of adults at work. The typical chief 
executive officer (CEO) in real corporate board rooms and in the media is male; he wears a suit and looks like a 
competent leader. The real elementary school teacher is female, dresses for doing school projects, and wears a 
pleasant expression; in the media of course, she is young and good looking, or reflects the kindly grandmother 
image. The roles are defined, the images are engraved in people’s minds. 

The socially constructed meaning, expectations, and opportunities based on gender begin with differing 
expectations for people, depending upon their sex at birth (Stem & Karraker, 1989). They are revealed in the way 
we groom boys for leadership positions while we teach girls to be submissive, in the way we emphasize the 
importance of male-dominated sports, in the way teachers respond to boys differently than to girls (Olivares & 
Rosenthal, 1992), in the way stereotypes are perpetuated in the media (Kilboume, 1990; Schwartz & Markham, 
1985), and in the way we recruit for jobs (Bern & Bern, 1973; Fidell, 1975; Rowe, 1990). They are revealed in the 
way we provide examples, exercises, and meaningful educational opportunities that boys can often relate to better 
than girls. It reveals itself in the grooming of boys for entire categories of jobs involving science, math, medicine, 
and politics. 
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Girls can achieve equally well in the aforementioned areas but have not been encouraged to do so until 
recently. And now the attempts are filled with remaining hurtles and barriers that must be overcome (Top, 1991). 
Meaningful instructional design practice must do more to attend to these matters and take an active role in 
encouraging girls (Van Nostrand, 1991). While not enough has been done and it is too late for many, instructional 
designers can begin to make amends for those girls who have yet to come through our nation’s school systems and 
workforce training programs. Instructional designers can make a better effort to provide experiences that girls can 
relate to, offering instructional opportunities that are not gender biased, and encouraging teachers to actively attend to 
issues of gender equity (Turkle & Papert 1990). 

Let us examine some ideas about groups, territory, belonging, and oppression, and relate that information 
to the visual display of information about using computers. Community is the joining together or grouping of 
individuals in society. People tend to seek out others that have something in common with themselves and to 
whom they can relate to comfortably. People form communities from both natural and constructed situations. 
Natural communities are those that happen as a result of nature, such as being bom a certain race, with a certain 
color hair, or as a male or female. Constructed communities are defined by boundaries that humans set by beliefs 
and interests. Religions beliefs and resulting congregations are once example. Another example is chosen activities, 
such as area of study, professional occupation, hobbies, leisure activities, participation in clubs, and so on. 

Communities vary by the intensity of the membership in the community and how society views their 
importance. Communities play a large and significant role in society. They not only define where individuals fit 
into society, but also help people to establish identities. They provide continuity and a sense of belonging to 
something greater than an individual realm. Communities give people purpose. 

People who are not members of a certain group can be left out, not necessarily due to being ostracized, but 
often due to sheer neglect, being viewed as the “others” or “them.” This is where the concern comes in for females 
and the internet. For members of our society, the consequences of being out of the group, or an other , are usually 
not extreme on the surface but the subtle biases levied by those in powerful positions over many years are 
cumulative and can be devastating in terms of the domino effect that result. From that perspective, there is a 
paradigm established from which to examine potential remedial actions to get more females involved in computer 
technology endeavors. 

Studies about females using technology often show that the percentage of females using technology is 
lower than the percentage of males using technology, yet they do not usually look at the social factor of natural 
group selection. Males are not necessarily trying consciously to keep females out of the computer domain, but they 
often exclude the females by forgetting to include them. Thus something like the casual conversations in which 
information is exchanged about computers can have a cumulative effect of leaving women in the dark about the 
computers. 

Many women who work with technology believe that the environment is hostile toward them (Turkle & 
Papert, 1990). Yet, in order to correct this neglect an omission of women from the computer culture calls for a 
change the attitudes and behavior of society toward women and technology. Yet gender differences in attitudes toward 
technology begin in the way that males and females are raised, thus reflecting the social expectations of individuals, 
family, friends, and society (Canter, 1979; Davies & Kandel, 1981; Eccles, 1987), so this becomes a rather large 
issue. 

Forcing males to change value systems will not necessarily help, but could actually do even more damage 
the relationship between males and females. Rather than insulting the technology-using male community it would 
be more productive to seek common ground on which males and females can begin to work together. One way of 
helping to establish this common ground is through imagery portrayed to the general public. Imagery that shows 
regular women in productive roles using technology in powerful ways would help. 

The existing male technology community was shaped by our society. Members of that community are 
different in many ways from typical members of society in that they have found the common ground of special 
interest in computer technology. Like other special interest areas, they cannot be forced to accepted others who do 
not seem to be a natural fit. If females have any chance of fitting into the computer technology community, then 
the males will need to be able to view the females as members of the community who have something in common 
with them, rather than as sexual objects, decorations, or servants to the male needs. 

This concept of building common ground begs society to examine the way it defines community. The 
deep and persistent problem of gender separation begins in early childhood, permeates the toy and fashion industries, 
sticks in the minds of teachers who separate boys and girls in to separate groups, and pushes its way into homes as 
parents succumb to societal pressures separate activities along gender lines. . If boys and girls are not allowed to 
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develop working relationships and friendships in early childhood, how can they be expected to overcome the barriers 
in later years? If they are bombarded with images of stereotypes about gender and technology throughout their lives, 
how will they be expected to overcome those stereotypes in later years? 

It is important to ask men if they are trying to prevent women from joining their community. Is there a 
social bias toward females that begins in childhood, or is there just the simple comfort of migrating toward your 
own community with its common bonds? If that is the case, then the early gender message that boys and girls 
receive become especially important and influential. 

While it might be true that boys and girls are in the same classes at school, it is also true that some 
teachers do a very efficient job of separating them within classroom groups. For example, “Preschool X” seems to 
have wonderful teachers but the entire group focuses on separating the children by gender. The class lists are hung in 
the hallway and distributed to parents not on the basis of who is in what class, but by gender, with all of the boys’ 
names listed at the top and all of the girls’ names listed at the bottom. When children break into groups it is by 
gender, and even on field trips, parents are assigned groups of children to supervise by gender (Knupfer, 1995-97). 

It seems that society offers opportunities to mix, yet at some point during our childhood years those 
opportunities are changed by parents and teachers who define them as taboo situations, or blindly ignore the 
opportunities for shared experiences and the consequences of segregation. What role the media plays in this is 
certainly complex. By the teenage years, males and females are strongly discouraged from intermingling by parents 
who fear sexual encounters. Are we to believe that males and females cannot interact in platonic ways? If that is a 
common belief, then perhaps it is a result of the way the media continues to influence people through its portrayal 
of men and women in stereotypical ways (Roberts & Maccoby, 1985). 



Print Media 

A representative sampling of magazines collected by several people revealed consistent patterns in the 
advertising about the new technologies. Men were portrayed in prominent positions within the advertisements and 
accompanied by messages to “work stronger, work harder, gain power, manage people, and gain career 
advancement.” The men projected professional images of people with great competence who were successful at 
work. The women on the other hand, were depicted in subordinate positions, serving others, and in roles that did not 
use the technology to accomplish their jobs or gain power or promotions. The women were usually wearing casual 
dress, often red in color, and seemed to be used as decorative graphics rather than in any meaningful way. 

One magazine for educators, contained only two technology advertisements that included people. Toward 
the center was an ad with two women teachers, one a middle-aged, grandmotherly-looking, English teacher and the 
other a young, slender, blond, graphic designer. There were the stereotypes, the overweight and dull-looking 
grandmotherly English professor and the sexy, young blond graphic designer. Unfortunately, the advertisement also 
depicted both women standing at the bottom of a stairwell, clearly doing nothing for their career success by using 
their laptop computers that matched their body builds. Yes, the grandmother held the heavier, boxy laptop, and the 
young women held the more slender laptop, while the advertiser claimed that the computers were matched to the 
needs of the two women. 

The inside back cover of that magazine carried the second image of a person, this time a male. In addition 
to being in a more prominent cover spot, the man depicted in the advertisement clearly was on top of the world as a 
result of using his laptop computer. He was freed of his ties to his office and was overlooking a beautiful body of 
water with lovely islands, working independently. Obviously, his laptop computer had liberated him from the 
physical confinements of his office space. 

Both advertisements were produced by the same company, both were in black and white, both promoted 
laptop computers, and both followed the stereotypical representation of men and women regarding computer usage. 
Further, the advertisers even included the typical media portrayal of women as either grandmotherly or young and 
sexy. What ever happened to typical-looking women? Why are they absent from the advertisements? 

Not one of the printed advertisements showed women achieving school or career gains due to using the 
technology. Why not? In contrast to the teenage girl wearing the red blouse and reading a book while sitting next 
to a computer, a similar advertisement depicted a teenage boy who was proclaimed to have discovered a prize-winning 
formula, obviously implying that he did so as the result of his use of computer technology. This contrast seems 
obvious. Why then would so many people not notice the imagery and continue to perpetuate it? 

There were some interesting advertisements for business travelers that focused on the great features within 
new models of wheeled luggage that could be rolled down airplane aisles. Those with male characters tended to 
emphasize pockets for computer equipment, maintaining a common statement of being functional within a 
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technological business sense. The advertisements with female characters focused on being lightweight and spacious 
enough for all of a woman’s weekend fashion needs. 

One luggage advertisement clearly depicted three servants who were packing their employer’s bags in 
preparation for a business trip. Although the reader could not see the employer, it was obvious that he was male 
from the collection of objects the servants had gathered. The most prominent feature of the advertisement was the 
computer equipment that fit so neatly within the luggage compartments. The advertiser wanted so much for the 
computer space to be noticed that the computer was given a much more important spot than the luggage itself. 
Perhaps most importantly, the advertisement clearly portrayed the message of the computer being a male machine, a 
very important male object within the collection of male clothes and supplies. 

An eye-catching advertisement for a computer server, depicted a young, slender, and very sexily-posed 
woman sitting on the floor and leaning on a red, distorted computer standing in the vertical position. The woman 
was purely decoration. The advertisement dripped with sexual innuendo. A subliminal image of male genitalia was 
super-imposed onto the upright computer, reflected in the white lighting that matched the woman’s white blouse that 
leaned against it. Such an advertisement certainly raises questions about what is being sold and under what disguise? 

From this sampling of printed advertisements, it is clear that females are depicted in technology 
advertisements much less frequendy than males. Further, when women are depicted, they are usually used as 
interesting decorations rather than as productive users of the technology. 

Television 

Television programming that reaches into most American homes. In television, women typically are 
depicted as feminine and showing their beauty, grace, style, sexual attractiveness, and subservience to men 
(Rutherford, 1994). Often they are depicted as beautiful yet not very intelligent, such as in Three’s Company or 
Married With Children. The imagery of feminism is defined by the television producers and advertisers, not by real 
women in real life. 

Consider the persuasiveness of televised advertising and the way it has influenced women to buy more and 
more cleaning products, believing themselves to be better homemakers if they clean better (Rutherford, 1994). But 
do these products actually liberate women from the household work by making it easier or do they tie women to the 
home by raising expectations for cleanliness and in so doing, keep women cleaning? 

Advertisers like to show women shopping, cleaning, cooking, and looking after others. The act of cleaning 
takes on a meaning of love, order, and conformity. Thus the image of the homemaker providing a haven of 
harmonious safety, regardless of the state of the outside world and its many threats, is flashed repeatedly before the 
eyes of the viewing public (Rutherford, 1994). 

On the other hand, men are associated with the outdoors, sports, cars and driving, relaxing, or entertaining 
at home (Rutherford, 1994). Roles within the work and home environments that command expertise or authority, 
such as recommendations for finances, insurance, making decisions for senior adults, or offering professional 
opinions, typically feature men. In addition, adult males are usually the announcers or authoritative voices for 
products, even for products mainly used by women. 

The messages of gender separation begin early, laced throughout society in many ways. For example, the 
television commercials within popular children’s programs feature the gratuitous sequence of two commercials for 
boys and two commercials for girls, not necessarily in any specific order, but certainly distinctive in gender imaging 
(Courtney & Whipple, 1983; Downs & Harrison, 1985; Kilboume, 1990; Lovdal, 1989; Macklin & Kolbe, 1994). 
The boys’ commercials are fast paced, outdoors and rugged, usually involve something on wheels, often depict 
aggression, employ music with a fast tempo and distinctive beat, and use camera angles that slant up, placing the 
boys in a dominant position within the picture. The girls” commercials are slower paced, indoors and sweet, usually 
involve dolls or fashion messages, often depict emotion and caring for others, employ musical sweet calm 
undertones, and use camera angles that slant down, placing the girls in the less dominant portion of the picture. 

The stereotypes reinforced by television carry over into daily life (Berry & Asamen, 1993; Berry & 
Mitchell-Kerman, 1982; Bretl & Cantor, 1988; Fidell, 1975). Bring this imagery to the real experience of entering a 
computer consulting office at a typical university campus. The consultants are typically male, speak in a language 
of their own, and if you can understand the language then you might understand the help that is offered. Of course, 
the quality of help you get might depend completely on the degree of assistance that the consultant feels like 
providing, depending on an instant first impression of your needs and your perceived capacity to understand the help 
you get. With all too much frequency, the consultants miss completely and talk down to female clients or speak a 
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language of acronyms and other technobabble that the females do not understand. Why? Where does this 
miscommunication start and what perpetuates it? 

As we consider the answer to this question, we must ask a host of related questions. For example, why do 
the commercials depict boys and girls separately, reinforcing the separation by sex at an early age that leads to 
separation by gender throughout life? At what point and for what reasons do the genders mix in the visual images 
depicted in advertising and in real life? What messages are given to males and females about turf, not mixing, and 
why? Is there common ground that can be found, and why is it not more frequently visited? As more emphasis is 
placed on using the visual media in schools, what implications does this separation have for males and females? 

Cyberspace 

As we reflect upon male and female domains, it is clear that computer technology has been dominated by 
males over what has now grown to decades. As computer skills become more important during this age of 
information and cyberspace activity, we need to think seriously about how gender stereotypes can influence not just 
the amount of time that people spend at computers, but the type of activities that males and females are exposed to 
when using computers. Further, we need to consider the types of messages that are carried to people about what is 
expected, appropriate, and acceptable.. 

Advertisers have invaded the electronic arena of the Internet and its popular graphic vehicle, the WWW. The 
Internet is clearly an environment that was structured by and for males. Now that the Internet is placing more 
emphasis on the WWW, will this richness of the visual communications environment be male turf like other 
technology areas or will the Web be utilized as easily by females? What societal factors will influence the outcome? 
So far, messages within our society that are reinforced by the mass media emphasis that youth and attractiveness are 
good, sex is important, and males and females have certain roles to play that are separate and distinct. 

Recent research reveals that men and women communicate differently on the Internet, and that they have 
different communication ethics. One study revealed that nearly 70 percent of the messages posted by men used an 
adversarial style in which the writer distanced himself from, criticized, or ridiculed other participants, and often 
promoted his own importance. In contrast women displayed features of attenuation, often hedging apologizing, and 
asking questions rather than making assertions. The women’s postings also tended to more personal both in terms 
of revealing thoughts and feelings, and being supportive of others (Herring, 1996). 

Although some people claim that electronic communications are anonymous and therefore invite 
participation of both males and females, that will not necessarily be the case if our on-line communicative style 
reveals our gender. For then gender differences, along with their social consequences, are likely to persist in 
computer-mediated networks (Herring, 1996). As a part of the on-line network that is fast growing, popular, and 
graphically oriented, the Web holds a prominent position, so it will have great impact on its users. If females are to 
use the web productively, then it cannot be restricted to the male domain. 

As we move into the new forum of Web communication, the public will once again have an opportunity to 
shape the messages about shared space and gender stereotypes. What will we see in the near future on the Web? At 
least one author has already portrayed the Web as male space, depicting the changing face of advertising, Web space 
that is, in a male hat, necktie, and business suit (Geracioti, 1996). Even though the majority of network users are 
males (Shade, 1993), females must be encouraged to learn skills and be provided with opportunities to have equal 
access to information, and be portrayed in productive roles using technology. Women will have a chance to stake 
out a space, but will it be obvious or will it be overshadowed by the male presence that started the Internet and 
continues to dominate computer culture? 

The answer to that question is certainly not simple, but people can influence educational applications of the 
Web by designing instructional environments that attend to the needs of the female population as well as those of 
males. Several authors have made suggestions about things that teacher and parents can do to improve gender equity 
in schools (Couch, 1995; McCormick, 1994; Olivares & Rosenthal, 1992; Ploghoft & Anderson, 1982). Those 
ideas can be transferred directly to any educational environment that does or does not use technology. To begin, 
parents and teachers can encourage reflective practice that makes adjustments to the needs at hand and considers ways 
to disassemble the walls of current gender segregation within our society. The information age brings the challenge 
of shifting responsibilities and one of the most important will be to find the common ground upon which males and 
females can find trust and build healthy working relationships that begin in childhood and carry on through adult life. 
It is no small challenge to overcome the images of mass media and years of stereotypes, but it can be done one step 
at a time. 
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ITV Educational Environments 

The deeply ingrained cultural stereotypes and practices related to gender continue to support instruction in 
its historical sense by constantly feeding the old system while stifling attempts to pay serious attention to gender 
equity (McCormick, 1994; Gornick & Moran, 1972). Materials developed for use in public, private, and military 
schools as well as instructional messages delivered to the public through advertising, television, public service 
messages, and the WWW continue to portray males and females in stereotypical ways, with males in dominant 
positions. This has indeed been carried forth into the gender messages portrayed to the public about using the new 
technologies (Knupfer, 1996). Common practice dictates that males will serve in leadership and authority roles 
while females serve in subordinate positions. We see it in the images of males and females in the media, and we see 
it in practice. Thus decisions about instruction are often made by males and need to be implemented by females. 

Integration of interactive television (ITV) into the regular course schedule is occurring in more instructional 
situations and in greater numbers of course offerings each year. If implemented appropriately, ITV can provide a rich 
and powerful learning environment that employs visual learning in a meaningful way. It can allow the sharing of 
visual resources across various groups of people, in different geographical locations. 

The reasons for the increased use of ITV in recent years are numerous, ranging from the practical needs of 
serving students to the desire to integrate technology into the learning process. Politicians are pushing for increased 
funding for and utilization of ITV by convincing people that ITV increases the interaction between teachers and 
students. The real reasons for implementing ITV into classrooms span the range from real curriculum needs to false 
needs that are driven by the technology push. 

The specific reasons for implementing ITV are varied and many for each situation, yet the common thread 
among all who implement ITV is that they claim to be seeking a better way to serve various educational needs. 
Many people believe that the most important advantage of ITV for small, geographically isolated schools is that ITV 
provides access to advanced or specialized courses that would not otherwise be available to them. Although ITV can 
provide enhanced curriculum opportunities and scheduling alternatives, the outcomes of using ITV are varied and 
complex. There is a dearth of information in the literature pertaining to the complexities of ITV’s attendant 
outcomes. 

While ITV is reputed to be successful and liberating in that it frees students from the restriction of place, 
and it is potentially disempowering as well. Students and instructors whose needs and preferences are ignored 
throughout the planning, implementation, and evaluation processes will not necessarily be able to successfully reach 
their educational goals by using the resulting ITV environment. 

Educational institutions are purposeful enterprises driven by power structures that are informed by openly 
endorsed agendas of both political and personal origin (Freire, 1970; Knupfer, 1993; Shor & Freire, 1987). Analysis 
reveals that leadership and decision-making is clearly dominated by males who hold nearly all of the leadership roles, 
while the people who implement the ITV are females (Kansas State Board of Education, 1995). Preliminary analysis 
of the various constituencies that guide ITV network operations yields a similar result. At first this seems 
inconsequential, but further thought raises questions about the situation. For example, consider whether students and 
faculty are best served by decisions that are made by the dominant, male administrative culture without input by the 
female teachers and students who use the facility. 

From time immemorial, males have been in decision-making roles within American homes, businesses, 
and educational institutions. Society has carried messages about gender stereotyping within instructional products 
and schooling practices, through the mass media, within social practice, and so on, thus perpetuating the situation 
complete with its advantages and disadvantages. Of course, there are good points to male leadership but there also 
are drawbacks. The disadvantages are likely to occur when decisions are made in isolation without regard to the 
female perspective. 

As we witness a new time in education, when people are calling for school reform and the implementation 
of technology all in one breath, there is an opportunity to ask once again, whether or not there is adequate 
representation of the teachers, mostly female, in the business of ITV. It seems that the role of women in relation to 
the new media has made little progress, possibly even less than that within the more traditional forms of schooling. 
Could this be because technology has been viewed as male territory? 

The complexities of this situation are enormous. Like society itself the complexities reflect the dynamics 
of different situations in different ways, among different individuals. An examination of the complex relationship 
between instructional planning and design, and gender reveals inequities that result from a persistent pattern of 
practice. Recognizing the result of those inequities can be easier than finding the causes and correcting the problem. 
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Inequities that result from the traditional practice of male decision making and female responsibility for 
implementation within the instructional system often go unrecognized because they emerge not just as a result of 
what has been done, but also as a result of what has been left undone. The neglect and omission of the female 
population reveal themselves in subtle ways on an individual basis, but as a collective result appear throughout 
society as something that begins in the home, and perpetuates itself through schooling and employment practices. If 
that were not the case, then there would be no need for recent efforts to attract girls into fields of study that are 
typically populated by males (Kable & Meece, 1994) and the number of distressing stories about females succeeding 
despite the myriad of obstacles (Aisenberg & Harrington, 1988; Clark & Corcoran, 1986; Frenkel, 1990; Gomick, 
1990) would no longer be told. 

Summary 

The real concern of this paper goes beyond the biases of the media representations, to the larger dimension 
of the people they influence, from the instructional designers to classroom practice, from teachers to students, from 
parents to children, an so on. Instructional designers can influence the entertainment industry, home market, school 
environment, and practices in business and military environments. Designers can accept the importance of their role 
in shaping the self concept and encouraging equitable access to job skills that later translate into life skills and wages 
commensurate with experience. A more positive and forward-thinking outlook on the role of females in our society 
can certainly do much to influence the drive and effort that is currently necessary for females to overcome the many 
obstacles in daily life. 

Instructional designers can influence educational practice by designing instructional environments that attend 
to the needs of the female population as well as those of males. They can encourage reflective practice that makes 
adjustments to the needs at hand. The information age brings the challenge of shifting responsibilities. Even 
through the majority of network users are males (Shade, 1993), females must be encouraged to learn skills and be 
provided with opportunities to have equal access to information. 

Until society gets beyond viewing women as second class citizens in stereotypical roles, then instructional 
designers will have a difficult job in educating the public. Yet it can be done. The first step is to educate 
instructional designers to attend to the needs of a pluralistic society. The second step is to encourage business, 
industry, government, and education institutions to include knowledgeable designers on their project development 
teams. The designers can produce text-based and mediated materials that attend to the needs of females as well as 
males. Further, they can provide better training for teachers and others who provide information to the masses. 
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Abstract 

This paper expresses different points of view about the relationships between males and females , the 
computer culture , the influence of mass media , and community. It cautions readers to understand the need for 
sensitivity to the male perspective and the need to address the issue of gender from within a societal context , rather 
than from a male versus female approach. 

Mass media images of our society reflect the stereotypes and the realities of gender tracking, often 
separating males from females. For example, one might think about a common media image of adults at work. The 
typical chief executive officer (CEO) in real corporate board rooms and in the media is male; he wears a suit and 
looks like a competent leader. The real elementary school teacher is female, dresses for doing school projects, and 
wears a pleasant expression; in the media of course, she is young and good looking, or reflects the kindly 
grandmother image. The roles are defined, the images are engraved in people’s minds, and they appear in various 
forms of artwork, entertainment, advertisements, productions, and educational materials. 

The cultural messages of gender separation begin early and appear throughout society in many ways. For 
example, the television commercials within popular children’s programs feature the gratuitous sequence of two 
commercials for boys and two commercials for girls, not necessarily in any specific order, but certainly distinctive in 
gender imaging (Courtney & Whipple, 1983; Downs & Harrison, 1985; Kilboume, 1990; Lovdal, 1989; Macklin & 
Kolbe, 1994). The boys’ commercials are fast paced, outdoors and rugged, usually involve something on wheels, 
often depict aggression, employ music with a fast tempo and distinctive beat, and use camera angles that slant up, 
placing the boys in a dominant position within the picture. The girls” commercials are slower paced, indoors and 
sweet, usually involve dolls or fashion messages, often depict emotion and caring for others, employ musical sweet 
calm undertones, and use camera angles that slant down, placing the girls in the less dominant portion of the picture. 

The stereotypes reinforced by television carry over into daily life (Berry & Asamen, 1993; Berry & 
Mitchell-Kerman, 1982; Bretl & Cantor, 1988; Fidell, 1975). Bring this imagery to the real experience of entering a 
computer consulting office at a typical university campus. The consultants are typically male, speak in a language 
of their own, and if you can understand the language then you might understand the help that is offered. Of course, 
the quality of help you get might depend completely on the degree of assistance that the consultant feels like 
providing, depending on an instant first impression of your needs and your perceived capacity to understand the help 
you get. With all too much frequency, the consultants miss completely and talk down to female clients or speak a 
language of acronyms and other technobabble that the females do not understand. Why? Where does this 
miscommunication start and what perpetuates it? 

As we consider the answer to this question, we must ask a host of related questions. For example, why do 
the commercials depict boys and girls separately, reinforcing the separation by sex at an early age that leads to 
separation by gender throughout life? At what point and for what reasons do the genders mix in the visual images 
depicted in advertising and in real life? What messages are given to males and females about turf, not mixing, and 
why? Is there common ground that can be found, and why is it not more frequently visited? As more emphasis is 
placed on using the visual media in schools, what implications does this separation have for males and females? 

The Internet is clearly an environment that was structured by and for males. Now that the Internet is 
placing more emphasis on the World Wide Web (Web), will this richness of the visual communications environment 
be male turf like other technology areas or will the Web be utilized as easily by females? What societal factors will 
influence the outcome? So far, messages within our society that are reinforced by the mass media emphasize that 
youth and attractiveness are good, sex is important, and males and females have certain roles to play that are separate 
and distinct. It is no wonder that many women who work with technology believe that the environment is hostile 
towards them (Turkle & Papert, 1990). 



125 



127 



Gender, Society and Technology 

While our society encourages boys to get messy, wrestle, and explore unknown territory, girls get subtle 
messages to keep their hands clean, play with their dolls, obey the rules, and often they are discouraged from taking 
science and math in school (Kantrowitz, 1996; Rowe, 1990). While boys and girls are equally interested in 
computers until about the fifth grade, after that point, boys' usage rises significantly and girls' usage drops 
(Kantrowitz, 1996). This is most likely due to increased sex-role identification at that age. By high school, students 
show clear gender bias in their attitudes toward technology (Shashanni, 1994). Surely the attitudes of the high 
school students will carry into adulthood through higher education and into the workplace. 

The neglect and omission of the female population from math and technology fields reveal themselves in 
subtle ways on an individual basis, but as a collective result appear throughout society as something that begins in 
the home, and perpetuates itself through schooling and employment practices (Rowe, 1990). If that were not the 
case, then there would be no need for recent efforts to attract girls into the study of math and science (Kable & 
Meece, 1994) and the number of distressing stories about females succeeding despite the myriad of obstacles 
(Aisenberg & Harrington, 1988; Clark & Corcoran, 1986; Frenkel, 1990; Gomick, 1990) would no longer be told. 

Although many scholars insist that great strides have been made concerning gender equity, it has not been 
enough and the subtle biases remain barriers to equal opportunity (Rowe, 1990; Rutherford, 1994; Savan, 1994; 
Schwartz & Markham, 1985; Signorielli, 1993; Tannen, 1996; Top, 1991). The real concern goes beyond the 
biases of any specific people, to the larger dimension of the teachers and students, instructional designers, artists, 
advertisers, administrators, and families that make up society as a whole. The evidence is clear in the games of the 
children, the classroom practices, the design of educational products and environments, the advertisements found 
within all forms of visual media, and the visual messages conveyed by the entertainment industry. 

Since the computer industry is relatively new, one would expect a more gender-diverse leadership. Yet that 
is not the case and women continue to struggle to gain knowledge, to gain credibility, and to achieve career 
advancement within the technology fields (Kantrowitz, 1996). Couple this with the existing difficulties faced by 
females who work on university campuses (Aisenberg & Harrington, 1988) and the dynamic becomes even more 
complex, especially as administrators begin to do such things as include degree of technology usage within employee 
evaluations. It would appear that many females will find themselves stuck, as they say, between a rock and a hard 
place. 

Many people are either unwilling or unable to see beyond their current practices, beliefs, and biases. The 
majority of decisions about technological purchases and utilization being decided by males, thus leading to situations 
that are structured for males users. Why do we not routinely see male administrators seeking the opinions of female 
subordinates prior to making decisions about technology that will affect them in their work. For example, why are 
more teachers not involved in fact finding prior to making important decisions that affect their classroom practices? 
If computer-supported multimedia is such an important part of our lives, then why do we not see more women on 
the executive boards of technology companies? 

The president of a technology company answered this question by saying that women have never been on 
the board of directors in his company and they are not acceptable candidates because they are not reliable. Why are 
they not reliable? Well, they will get pregnant and then they leave. When told that the women might come back 
quickly to jobs that reward them, the executive said the real reason was that women could not be trusted. Why not? 
Because they do not fit and it would not be appropriate to discuss important business with them (Larsen, 1996). It 
sounds outrageous but is it really that far from the truth of what commonly occurs in our society? Is it isolated to 
corporate board rooms? Is it part of the computer culture? 

Clearly, there is evidence of different types of usage and different attitudes toward computers between males 
and females. Men tend to be seduced by the technology itself rather than what they can do with it in a practical sense 
(Kantrowitz, 1996). Like bragging about fast cars, men often brag about the size and speed of their computers and 
software. On the other hand, women tend to focus on the utility of the machine rather than its glamour. They do 
not care about what is on the inside nor what makes it work, but are very much concerned that it function 
sufficiently well to meet their needs (Kantrowitz, 1996). 

The difference can be stated simply as a male tendency to focus on the tool itself and a female tendency to 
focus on the utility of the tool. While men tend to think of computers as powerful ways to extend their physical 
limitations, women tend to think of them as a means to an end (Kantrowitz, 1996). Men want to force computers to 
submit while women just want computers to work (Tannen, 1996). Thus we see a difference in the relationship 
between people and computers based on gender. The media capitalizes on those differences and further widens the gap 
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by concentrating on or exaggerating stereotypical roles in the way people are portrayed in computer clip art (Binns & 
Branch, 1995) and in advertisements about computer technology (Knupfer, 1996). 

The literature further reveals that women are not well represented in the new generation of high technology 
occupations. Changing that is important because society can ill afford to waste half of its innate talent, nor can it 
justify wasting of talent due to gender-based access to opportunities. Discouraging half of our population from 
entering the high technology fields is a practice that can only hurt our society. Rather than thinking about what is 
good for which gender separately, the focus should be on thinking about what is good for society as a whole. Instead 
of arguing for equal rights for women in a way that pits women against men, it would be more beneficial to 
approach this as a societal problem that must be solved by working together. This distinction is crucial and it can 
be supported by the way in which women and men are depicted in the mass media. 

Importance of Communities 

Although research on computer-mediated communication dates back to the early days of computer network 
technology in the 1970's, it is only recently that researchers have begun to take into account the gender of the users. 
Recent research reveals that men and women have recognizably different styles in posting messages to the Internet, 
and that they have different communication ethics. One analysis of listserve discussions revealed that 68 percent of 
the messages posted by men used an adversarial style in which the writer distanced himself from, criticized, or 
ridiculed other participants, and often promoted his own importance. In contrast women displayed features of 
attenuation; they hedged, apologized, and asked questions father than making assertions. In addition, the women’s 
postings tended to reveal a personal orientation, revealing thoughts and feelings, interacting with and supporting 
others (Herring, 1996). 

Although some people claim that electronic communications are anonymous and therefore invite 
participation of both males and females, that will not necessarily be the case if our on-line communicative style 
reveals our gender. For then gender differences, along with their social consequences, are likely to persist in 
computer-mediated networks (Herring, 1996). As a part of the on-line network that is fast growing, popular, and 
graphically oriented, the Web holds a prominent position, so it will have great impact on its users. If females are to 
use the web productively, then it cannot be restricted to the male domain. Let us examine some ideas about groups, 
territory, belonging, and oppression, and relate that information to the visual display of information about using 
computers. 

Let us establish a straw man that represents the way that many people believe an identifiable group of 
people are hurt. In the case we are presenting, the injured or oppressed group is females. Which group is hurting 
them and why? Since the oppressed or victim group is identified based upon sex, not gender, it follows that the 
other group, the oppressors, must be composed of males. Logically, it follows that since females do not have equal 
representation in the high technology fields, remedies are needed that modify the hurtful behavior of the males so that 
the females’ rights to equal opportunity are respected. 

This process is based upon two fundamental ideas; the canon of individual rights and the process of 
“othering.” Both of these need a clear airing. They are commonly misunderstood and misapplied. The canon of 
individual rights simply declares that individuals have rights. These rights are both inalienable and inherent. The 
United Nations General Assembly passed a statement of these rights in its 1948 document, Declaration of Human 
Rights , and reaffirmed the document within the last few years at Helsinki. In that document, certain things were 
declared to be rights by birth. The concept of rights by birth is quite simply wrong. It is wrong because it leaves 
out the modifying clause, “being bom into a society.” Being bom into a society , each individual has certain rights 
by birth. If rights exist only within the context of a society, then that is a crucial omission. First, rights exist only 
within the context of a society. Second, rights are only violated through willful acts. “Othering” is an 
interesting concept that is the dark side of community. Community is the joining together or grouping of 
individuals in society. If an individual can say that a member of a community is, in some meaningful way, “just 
like me” then that individual is also a member of that community. 

Communities come in two classes, natural and constructed. Natural communities are those that happen by 
birth or mishap. Skin color is one such community, blindness is another. Constructed communities, which are far 
more numerous, are largely voluntary. Religions are a good example. Soap opera viewers are another. Some 
constructed communities are based upon natural ones, for example, a cohesive group based on ethnic origin that has 
come together to recognize common interests based on that origin. 

Communities vary by the intensity of the membership in the community and how society views their 
importance. Communities play a large and significant role in society. They not only define where individuals fit 
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into society, but also help people to establish identities. They provide continuity and a sense of belonging to 
something greater than an individual realm. Communities give people purpose. 

But community has a dark side which is called othering. Othering is the opposite of community building. 
The meaning of being an other varies with the community in question. It is the group of people who can be harmful 
or destructive to a community. For the double jointed community, once you get past elementary school, being an 
other makes very little difference. For a Moslem in Bosnia, a Hutu in Burundi, an urban dweller in Kampucheia, a 
Jew in Nazi Germany or, at about the same time, a Ukrainian in the Ukraine, being an other had very serious 
consequences. 

Othering and Common Ground 

For members of our society at present, the consequences of being an other are usually not extreme on the 
surface. However, the subtle biases levied by those in powerful positions over many years are cumulative and can be 
devastating in terms of the domino effect that result. From that perspective, there is a paradigm established from 
which to examine potential remedial actions to get more females involved in high technology endeavors. 

The basic premise of most studies about females using technology is that because the percentage of females 
using technology is lower than the percentage of males using technology, something is acting to keep females out. 
A case can be made that women’s rights are being violated and remedial action is required to change the technology 
environment so that women are welcomed. That means there is a need to change the attitudes and behavior of 
society toward women and technology. Yet gender differences in attitudes toward technology begin in the way that 
males and females are raised, thus reflecting the social expectations of individuals, family, friends, and society 
(Canter, 1979; Davies & Kandel, 1981; Eccles, 1987). 

But changing the society is something that cannot be done easily and must be done carefully. Remember 
that the canon of individual rights is based upon the premise that having rights will promote and protect society, not 
damage it, so the technology-using male community should not be damaged in the process. Instead there must be a 
way to work together to find common ground between the male and female communities. One way of helping to 
establish this common ground is through imagery portrayed to the general public. Imagery that shows regular 
women in productive roles using technology in powerful ways would help. 

When communities are defined, others become the enemy. Those others are individuals. They came to be 
who they are by their belonging, and not belonging, to a variety of natural and constructed communities. Perhaps 
their communities are dysfunctional, perhaps they are hostile to people unlike themselves. However they came to 
be, the fact is that they do exist. The existing male technology community was shaped by our society. Members of 
that community are different in many ways from typical members of society and those differences became a part of 
the “just like me” criteria. There are reasons why the communities are as they are. Simply wanting to change, 
simply demanding females’ right to inclusion will not work. Instead, the way to get females integrated into high 
technology fields is to find a way to get the males and females to be able to say together that they have some 
common ground within the technology community. That common ground cannot depict females as sexual objects 
nor decorations nor servants to the male needs. 

This concept of building common ground begs society to examine the way it defines community. The 
deep and persistent problem of gender separation begins in early childhood, permeates the toy and fashion industries, 
sticks in the minds of teachers who separate boys and girls in to separate groups, and pushes its way into homes as 
parents succumb to societal pressures separate activities along gender lines. If boys and girls are not allowed to 
develop working relationships and friendships in early childhood, how can they be expected to overcome the barriers 
in later years? If they are bombarded with images of stereotypes about gender and technology throughout their lives, 
how will they be expected to overcome those stereotypes in later years? 

It is important to ask men if they are trying to prevent women from joining their community. The male 
authors of this paper remembers his experience at one of the top technical universities in the United States. He 
recalls that there was absolutely no community with women. While growing up he had a ham radio, model 
rocketry, model trains, photography, and so on, but no exposure to girls on a social basis. There were virtually no 
girls involved in any of his activities. Why not? Were his parents narrow minded in this regard? Was his 
community social structure narrow channeled in this regard? Did he never explore beyond the boundaries of what 
came in his direction and asked to join in his current activities? In the days of his childhood, it was considered 
improper for a girl to extend the first invitation to a boy. For what must be a host of reasons, girls were not there 
and so the experience of working with girls did not exist in childhood years or in college. He says, most male “high 
techies” do not hate women. They love them, fear them, and perhaps have no clue how to talk to them, but they 
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certainly do not hate them. Women have just never been part of the male world and therefore there is a difference 
perspective. 

You might ask why he did not consider the primary school experience to be a mixture of males and females. 
After all, boys and girls do participate in joint physical education classes up to a certain grade, dependent upon the 
community and the school. While it might be true that boys and girls are in the same class, it is also true that some 
teachers do a very efficient job of separating them within classroom groups. For example, “Preschool X” seems to 
have wonderful teachers but the entire group focuses on separating the children by gender. The class lists are hung in 
the hallway and distributed to parents not on the basis of who is in what class, but by gender, with all of the boys’ 
names listed at the top and all of the girls’ names listed at the bottom. When children break into groups it is by 
gender, and even on field trips, parents are assigned groups of children to supervise by gender (Knupfer, 1995-97). 

In contrast, the female author of this paper does remember some exposure to the other gender during her 
childhood years. She remembers the neighborhood play and mixed gender acceptance of childhood. She also recalls 
that it was self initiated between the individuals involved and not assisted by teachers or parents. She remembers the 
distinct gender groupings of chldren when under the supervison of adults, such as at school or at birthday parties, and 
being called a “tomboy” by adults who observed her mixed gender activites and interest in sports. 

Males and females can play together and work together in productive ways. It seems that society offers 
opportunities to mix, yet at some point during our childhood years those opportunities are changed by parents and 
teachers who define them as taboo situations, or blindly ignore the opportunities for shared experiences and the 
consequences of segregation. By the teenage years, males and females are strongly discouraged from intermingling 
by parents who fear sexual encounters. Are we to believe that males and females cannot interact in platonic ways? 
If that is a common belief, then perhaps it is a result of the way the media continues to influence people through its 
portrayal of men and women in stereotypical ways (Roberts & Maccoby, 1985). 

As we move into the new forum of Web communication, the public will once again have an opportunity to 
shape the messages about shared space and gender stereotypes. What will we see in the near future on the Web? At 
least one author has already portrayed the Web as male space, depicting the changing face of advertising, Web space 
that is, in a male hat, necktie, and business suit (Geracioti, 1996). Even though the majority of network users are 
males (Shade, 1993), females must be encouraged to learn skills and be provided with opportunities to have equal 
access to information, and be portrayed in productive roles using technology. Women will have a chance to stake 
out a space, but will it be obvious or will it be overshadowed by the male presence that started the Internet and 
continues to dominate computer culture? 

The answer to that question is certainly not simple, but people can influence educational applications of the 
Web by designing instructional environments that attend to the needs of the female population as well as those of 
males. Several authors have made suggestions about things that teacher and parents can do to improve gender equity 
in schools (Couch, 1995; McCormick, 1994; Olivares & Rosenthal, 1992; Ploghoft & Anderson, 1982). Those 
ideas can be transferred directly to any educational environment that does or does not use technology. To begin, 
parents and teachers can encourage reflective practice that makes adjustments to the needs at hand and considers ways 
to disassemble the walls of current gender segregation within our society. The information age brings the challenge 
of shifting responsibilities and one of the most important will be to find the common ground upon which males and 
females can find trust and build healthy working relationships that begin in childhood and carry on through adult life. 
It is no small challenge to overcome the images of mass media and years of stereotypes, but it can be done one step 
at a time. 
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Abstract 

A interest in distance education increases , man instructors are experimenting with list serve discussion 
within their graduate courses. This paper seeks to include the students' perspective about the discussion list activity 
and offers an analysis of the value of the activity. Analysis areas of focus include the content, form, styles, and tone 
of the list discussion as well as perceived instructional value, patterns of gender influence, issues of power and 
control , and suggestions for improved future activity. 

Interest in distance education is increasing and many instructors are experimenting with different 
communication techniques within their courses (Willis, 1994). Part of the appeal of distance education is the result 
of access to e-mail and similar computer-supported communication. As more people become involved with the 
distance activities, there is a need to examine and evaluate what works well for instructional purposes in various 
situations. 

In an effort to stimulate ideas and discussion beyond the typical classroom experience, six instructors of the 
Educational Technology at four different universities initiated a discussion group for 52 graduate students within their 
various classes. The classes were all distinctly different, but had common interest areas related to the general field of 
study. The instructors selected four theme topics that could carry across the usual boundaries of specific course 
content and time frames in to a general discourse about educational technology and distance education. The four 
theme were regulatory issues, instructional design, sociopolitical issues, and technologies. 

Organization and advance planning were critical. While the faculty members integrated the on-line 
discussion differently in their classes, they all required students participation. A portion of the student’s grades were 
based upon the quality of their participation in the on-line discussion . Instructors each provided guidelines to their 
own students about how to actively participate in meaningful discussions. For example, in one class two students 
took turns moderating the discussion each week, and in another class one students took the responsibility for 
moderating the discussion all semester. 

Each class of students was assigned one of the four theme topics to moderate or focus on throughout the 
duration of a semester. Students carried on a lively conversation and the instructors analyzed the listserve activity 
from their perspective, making suggestions for improvements as appropriate. In addition, the instructors surveyed 
the students for their reactions to the activity. 

The Problem 

Although the instructor evaluation of the listserve activity is valuable, it does not truly account for student 
reactions to the events beyond what the instructors believed was important. This paper seeks to include the students’ 
perspective about the discussion list and is offered as a student analysis of the value of the activity. No matter how 
valuable the instruction seems to be from an instructor’s perspective, it will fall sort of expectations if it does not 
meet the needs of the students. 

For this study, all students in one class joined together to analyze the list activity. They assessed their 
reactions to the discussion list and analyzed the actual content of the discussions as well. Using elements from 
reader response theory, they evaluated the content, form, style, and tone of the list traffic, looking for patterns of 
communication that emerged. They found various patterns within the list correspondence and have made 
recommendations for improved future activity based upon those patterns. The students also provide an in-depth 
qualitative analysis of personal reactions to the activity and have suggestions based upon their experiences. 

Instructors believed that organization, collaboration, and flexibility were the key ingredients in hold student- 
led on-line discussions, but they failed to recognize the importance of the students’ feelings, reactions, and responses 
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as the semester evolved. As the participants, the students shaped the course of the discussion, responding or not 
responding as they chose, elaborating in areas of interest, and debating when differences of opinions emerged. 

Without collaboration of both faculty and students, the discussions would have been less successful. The 
collaborative approach could be described as instructor-initiated, student-to-student dialogues. Instructors had agreed 
to introduce the listserve to the students but to then stay out of it. Instructors intervened in the discussion very little 
and when they did so it was with the purpose of providing guidance an stimulating discussion. Instructors 
encouraged students to develop collaborative techniques for moderating and promoting in-depth discussions. At one 
point in time when a sensitive issue was raised, one student’ “flamed” another, thus changing the course of the 
discussion. The “flamee” discusses in detail what led to the flame, how it was handled without instructor 
intervention, what affect it had on the group discussion, and what affect it had personally. 

Students noticed that even though they were instructed to introduce themselves form a student role without 
the benefit of job titles and so on, some students identified job titles anyway and thus established a framework for 
exerting power and control over the other participants. Moderators sometimes changed the true flavor of a discussion 
when making a weekly summary, and although the summaries were viewed as helpful, they also had the potential to 
be distorted. What follows is some analysis of the discussion list from the students’ perspective. They comment on 
the content, form, style, and tone of the list discussion as well as perceived value, patterns of gender influence, 
issues of power and control, and suggestions for improved future activity. This commentary is timely information 
that is needed to guide the design and practice of distance education efforts. It will be particular helpful as 
universities on different campuses and with different courses join in future collaborative efforts. It is important to 
point out in this case, that all classes had different content and reading materials, with the four common threads of 
discussion being viewed as overall, general themes that could carry across a full program of study, but not be specific 
to any one class. 

Students Reactions on Survey 

Although the full survey will be analyzed in a different paper that the faculty are writing, it will be helpful 
to note here that the course started out with 52 students. After a time, one class of students got frustrated at the level 
of discussion and dropped out, bringing the number down to 44 students who participated on a regular basis. No one 
outside of the class that dropped was told that the class droppped out and so it simply appeared as it there was less 
traffic on the discussion list. Because this class had user ideas that came from the same university as one of the 
other classes, no one else could know which classes they were in within that university. We found out after the fact 
that the discussions were at too high of a level and that the students who had dropped out were in an introductory 
instructional design course. They were not comfortable with the level of discussion and believed that they could not 
understand it nor contribute to it. This is undertandable since at least one of the other courses had an advanced 
distance class enrolled. 

Of the 44 remaining students, 29 responded to survey that was distributed at the end of the semester and 
returned after the class had ended. Of those 29 students, 17 did not feel a sense of community with the group, 12 of 
them did. The large majority of students believed that the listserve experience helped them gain knowledge about 
distance education in general, listserves, and instructional design. 

Student Analysis 

Some students participated frequently in the discussion, while others did not particpate at all. There are 
several factors that might influence the message frequency. Some of them include: 

• Course requirement for X number of messages 

• Current threads 

• Facility with the technology 

• Access to the technology 

• Demeanor of participants within a particular thread / language used 

• Day of the week / period within the semester 

• Gender 

Problems within the Listserve: 

1. Posturing within one’s introduction. Participants were admonished to drive away from titles, credentials, 
etc. and focus on interest areas within their introductions in order to avert the development of a hierarchy within the 
TCLASS community. 
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2. Instructors should have indicated up front that the listserve contents would likely be analyzed. Hence the 
necessity of following message identification and content protocols would have been reinforced. This would have 
made it less troublesome in following threads and in contributing to them. As it was, the load of separating 
mislabeled messages was daunting. 

3. Lack of facility with the technology for some users. 

4. Lack of access to the technology for some users. (Time or Cost) This contributed to participants’ latent 
responses to entries that had already been exhausted or otherwise evolved to some other topic. 

5. Participants responding negatively to others rejection of any particular idea or to radically divergent shifts 
away from the original entry. 

6. Participants responding negatively to inaccurate interpretations of questions or comments. This led to a 
brief interaction that ended with one participant (JLE) posting no entries after about one month. 

7. Kinship between participants at various sites lead to discussions between themselves thus leaving others 
isolated from the discussion. There is evidence of electronic discussions going on individually between participants 
outside of listserve situation. 

8. Lack of emotional maturity or intelligence and professionalism by some participants was exhibited by 
defensive posturing. Inappropriate word choice (use of “you-language”) and name calling. This was docuemented 
among the males. Others expressed their dismay in various ways. Two people indicated that they were no longer 
interested in continuing as participants and one posted guidelines for constructive listserve activity. 

9. Discussions were often merely truncated without closure. Discussions were characterized by polarity. 
Support for ones’ commentary/opinion were too frequently drawn from personal observations and preferences as well 
as second party insight. References to research were uncommon. There was no accumulative summary directing 
future research. 

10. Relevant topics were ignored or recognized only in passing. Foremost, credentialism was mentioned and 
then dismissed. The issues of accreditation and credibility should be consummate issues. Without endorsement by 
some esteemed agency, be it academic or professional, the student -aka consumer - is at the mercy of the 
marketplace. Additionally, what curriculum can be delivered successfully was not addressed. What can we offer 
given the technical and financial limitations within which we operate? What should we offer given these same 
limitations as well as within the context of meeting learner’ various needs. Spinning off of the latter, we only 
superficially addressed student support services and the issues surrounding the topic. How do we justify what we 
offer if we cannot or will not provide essential student support according to learners’ expectations? How do we 
honestly assess the cost-benefit ratio? How then do we project fiscal needs and attendant fee structures? Early on, 
classroom-technology layout was touched on briefly and its implications superficially explored. System Design on 
any scale was only nominally explored. Methods for system evaluation were ignored. It would seem that the cost 
benefit ratio, student persistence, and grade assignments are “good enough” barometers of system efficiency. 
Additionally, we never sought to compare the various modes of distance education. What parameters influence the 
teaching-learning interaction across full-motion ITV (analog or digital: DS-3 or ATM), compressed video 
teleconferencing (ISDN-based PictureTel or pots-based C-Me-C-You), and Web-based courses. What assumptions 
and referents guide our adoption of any particular mode? Which features are universal and which are unique? Do we 
attempt to emulate the traditional classroom under the assumption that that is the optimal condition and because of 
our esteemed traditions or do we challenge that thinking and adopt delivery and service protocols consistent with our 
findings? 

11. It appears that participants, for the most part, did not anticipate the types of problems inherent of 
community development. It is evident that they initially looked to the instructors to mitigate problems with 
messaging logistics and possibly in resolving sore points. Participants of the list activity were reticent to grasp the 
enormity of establishing a community in which geographical and philosophical cliques already existed and within 
which a cadre of “authorities” allowed group dynamics to prevail. The evolution of TCLASS social norms; the 
groping for a sense of community at once with the establishment of individuality; looking to the instructors to 
resolve confrontational episodes; and ultimately holding one another accountable, albeit in a few instances, 
unprofessionally. 

Problems with Listserve entries: 

Entries to the list were sometimes confusing to follow. At the time of the list discussion, this served to be 
somewhat of an obstacle to smooth discussions, but it is unlikely that the students realized that these problems were 
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occurring or viewed it as anything more than irritating. During the analysis of the list logs, it became apparent that 
the majority of problems seemed to arise from a few main areas as noted below: 

• Identification - who from where. 

• Inconsistent use of Topic Headings. 

• Messages that combined Topic Headings 

• Sporadic messaging. 

• Alien entries - Distance Learning class outsiders and other messages passed along. 

While it would be undesirable to police or censor messages to the list, it would be helpful to find a way to 
make improvements in those areas. One way of doing that is to have the instructors moderate the list more closely 
and actually adjust material as appropriate. Since the instructors intended this list to be for the students, and took a 
hands-off approach in general, this would be out of character with the intention of the list. 

An alternative that is within the character of the list’s purpose, is to have the students take a more active 
role in moderating their own list activity. The instructors could provide more training for the students prior to 
starting the list activity, and then modify the expectation for students to moderate the list activity. For example, 
students could rotate moderating the list for the others, with higher expectations of what they will do. They were 
supposed to be doing this, but will need more knowledge and better incentives to make it happen in a better way. 

Recommendations for Future Listserve: 

Students who analyzed the list logs and the value of the activity, did not always agree on the interpretation 
of the experience, but it would be fair to say that they may very well have learned more from that analysis process 
than from their initial participation in the list. They suggest that improvements could be made in future listserve 
activity for any collaborative situation, by attending to two major areas, including establishing a sense of 
community and improving general communication. 

1 . Guidelines for establishing a community. 

Establish them early on and model them in order to continually reinforce them. In fact these should be 
discussed at each site prior to listserve involvement. Furthermore, ways of identifying potential problems and how 
to allay them should also be dealt with prior to listserve involvement. In order to do this, all instructors will need to 
be involved in the planning process well in advance of the class and guidelines established for the students. Students 
will do better with more active participation of the instructors at the onset and throughout the semester. 

2. General communication. 

Can we control these problem sources? There will always be those who have a need to tell us more than 
any one of us needs to know or wants to know. Inappropriate word choices, intentional or not, will always emerge 
when ownership or validity of ideas are challenged. Some people are stuck in a you-language mode having yet to 
attain the skills of elevating a dialog to an impersonal plane in which it is the idea that we dissect and not the writer. 
It seems a lesson from “Get Over It - 101” and one from “Assertiveness & Tact - 102” would go a long way. 

Recommendations for Listserve Analysis. 

1. Readers must establish a concrete set of codes accounting for messaging problems that confound data 
extraction. Adaptations to this set of codes should be made before coding begins in order that re-coding is not 
necessary. General topic headings must be few and interest sub-headings must be discreet. (It would help if 
TCLASS participants would have used accurate headings instead of what was merely convenient. 

2. Instructors for the course must establish under what conditions participation/membership is allowed and 
what messages shall be posted to the list and not violate those criteria. If an item is in doubt let the writer of the 
message post that message to the individuals toward whom it is aimed or post it to another list. The Listserve 
Moderator may then provide a pointer to the Listserve participants. 

3. Readers must establish under what conditions a message will be deleted from the analysis. Alien messages 
which engaged TCLASS participants in discussions were allowed because they often served as impetus for a 
controversial idea or an new thread. Alien writers having no connection to the group beyond a passing interest or 
request for information should be discarded from the analysis. 

Summary 

Although there were several problems that emerged from this activity, the participants have learned that 
with some additional efforts the situation could be improved. One of the things that is important is to realize that 
we learn not only from what goes well, but from our mistakes. Both faculty and students made mistakes throughout 



the semester and it is important to continue this effort of open communication to help improve the strategies that are 
used if this activity should be repeated. It would be very helpful to have students who analyzed this list activity pass 
along their suggestions to future groups of faculty and students who are planning to engage in list collaboration. 

While listserves within a specific class can more natural follow the course of events related to the class, the 
addition of different class topics at different universities complicates the situation. In such situations, the meaning 
of the list communication needs to be artificially stimulated at the being of the activity, for it is not likely to emerge 
by itself.. After all, students are very busy and have everything else in their lives to attend to prior to something 
that appears to be an “add on” to their class. In the future, we recommend that a definite structure be applied to such 
list communication and even though the instructors want the students to assume the lead, the students will welcome 
the instructors’ commentary to the list as well. While it is true that students will be intimidated by faculty 
comments that are counter to their own, they can benefit from faculty participation in the list. Both students and 
faculty could help to establish the guidelines for list operation, thus helping to create a sense of democracy while at 
the same time providing more meaning to the list by helping to establish a sense of community that reaches across 
campuses at the beginning of the semester. 
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Abstract 

An analysis of instructional products about creating web pages along with a sampling of home pages 
created by students , teachers , and others leads to suggestions for creating web pages from the perspective of visual 
communication and aesthetics. This paper examines several types of home pages including educational, commercial, 
informational, publications, and personal sites. 

Introduction 

In this popular time of electronic communications and emphasis on using the internet, more and more 
people are creating home pages. The power of the of the World Wide Web (WWW) is in the abiliy to use visual 
communications in a way that is not possible with the standard text of the Internet. Some say that a revolution is 
taking place — a digital revolution and the “most crucial task before us is not putting in place the digital plumbing 
of broadband communications links... but rather one of imagining and creating digitally mediated environments for 
the kinds of lives we want to lead and the sorts of communities we want to have” (Mitchell, 1996, p. 5). One 
alteration of this revolution is the digital environment of the graphical user interface (GUI). This environment, once 
static and linear, has changed. Users relate mutually to screens that are visual, interactive and non-linear (Jones, 
1995). Another transformation is the manipulation of images. Electronic tools allow the user to capture, display, 
distort, enhance, store and print images. These “developing technologies... blur distinctions between actual and 
represented reality...” (Stieglitz, 1995, p 22). 

The Internet, a global network linking millions of people world wide, is a digitally-mediated environment. 
The Internet is made easily accessible through WWW browsers. These interactive browsers, i.e., Netscape, Internet 
Explorer, Mosaic, include informational displays and access tools presented to users in the form of web pages. 
These pages, created in a computer language called HTML (Hypertext Markup Language) contain hyperlinks - 
graphics, buttons, icons, images, colored text, animation, video — that allow the users to browse the contents of the 
page, search and retrieve information, and link to other web pages around the world. Recent developments in on-line 
communication tools provide the ordinary user the ability to create interactive pages and offer course delivery on the 
WWW. “The technology to transfer and transmit graphic and other forms of visual communication instantaneously 
around the globe has made available for communication, a large audience, which is often visually illiterate” 
(Saunders, 1994, p 186). 

Purpose 

The purpose of the study was twofold: (1) to look at the design, aesthetics, and functionality of educational 
and non- educational web pages from the perspective of visual literacy and, (2) to evaluate printed and on-line 
materials that are used as resources by professionals and nonprofessionals to create these web pages. These “how to” 
manuals were evaluated for their discussion of good screen design, the use of graphics and icons as communication, 
backgrounds, hypertext, linking, and overall understanding of publishing on the WWW. 



The web pages were divided into major categories and sub-categories. The educational categories included K-12 
schools and colleges/universities. The non-educational categories included commercial, publications and 
communications, informational and cultural, and personal web pages. 

The subject of images, graphics, and icons is critical. From a technical aspect, if an image is memory 
intensive and being viewed with a low speed modem or limited machine capacity, the images load slowly. In 
creating web pages, the designer must convey to the user ways to access hyperlinks. This is sometimes done 
through icons, images, or text that act as interactive buttons. When the user clicks on these hyperlinks they are 
connected to new information, other web sites, or further resources. To be interactive the buttons must be properly 
coded in HTML, thus, guiding the user. 

The “business of creating web sites” is rampant. Many services, both on-line and printed references offer 
“how to” construction of web pages. However, little thought is given to the overall image of the page. Usually the 
focus is on the HTML language. 

Background 

The explosion of the WWW as an informational tool on the Internet (Descy, 1994) provides students and 
teachers, at all levels of education, with up-to-date information and continuous access to resources world wide. In 
effect, the Internet is a virtual library of information (Doyle, June, 1994). 

Most recently, the development of electronic utilities provided through WWW links, growth of software 
that contains HTML readers, and publishing of on-line and printed technical manuals have provided ordinary 
computer users with the ability to easily access, create, and publish web pages. In particular, K-12 educators have 
begun to create web pages as a way to provide information about their schools and classrooms, display student work, 
provide links for student-generated research, and link with other schools to allow students to collaborate with one 
another in learning activities i.e., creative writing and problem solving. 

Colleges and universities also use web pages as marketing strategies to provide information about the 
programs and courses they offer, as a way for students to apply to these programs or to register for courses. In 
addition, web pages are being created by college and university professors to offer courses via the WWW. 

The increased ability to create web pages that use graphics, icons, and images to communicate a problem 
comes at a time when “our educational systems have reduced the number of courses in art education and.... the 
availability of courses in visual literacy that would advance seeing and interpreting what is seen are only recently 
finding support in the educational system” (Saunders, 1994, p 187). Thus, many educators and students creating 
web pages have had little guidance concerning the visual design elements. The myriad of technical manuals provide 
information about how to create web pages but they don’t specifically address the necessary design elements that will 
help the web page communicate clearly and appear aesthetically pleasing. 

Visual design is a significant factor in the development of web pages. The “...visual medium is. ..superior 
because it offers structural equivalents to.. ..objects, events, relations and visual elements that can be “organized 
according to readily definable patterns” (Amheim, 1969, p. 232). 

Visual representations in the form of icons, graphics, and images are defined in two- and three-dimensional 
space as compared to the one-dimensional space of text (1969). According to Saunders (1994, p. 186-187), these 
visual representations (graphics) function as a communication tool because they are: 

• based on things that are seen 

• appeal to our emotions as well as intellect 

• convey a single overwhelming meaning or message 

• communicate information 

• comprise a language and grammar of which most of us are ignorant 

• are not designed to encourage feedback from the audience 

Effective web pages contain good screen design, graphics, icons, background images, image maps, and 
hypertext. The construction of web pages involves an understanding of the use of these elements. This 
understanding is inherent in the aesthetic and technical function of informational and form design as well as graphical 
user interfaces. 

“Good screen design uses the visual to portray the message to the reader.” Text with white space, the 
position of graphics, mixture of text and graphics, consistent placement of navigational buttons, standard icons, and 
borders act as visual aids to the user (Knupfer, 1994, p. 216). 

Graphics are used to make visual statements — to communicate visually. Visual statements are “an object 
or collection of objects created or assembled for the purpose of being seen and experienced” (Curtiss, 1987, p. 4). 



Visual statements contain visual symbols that include shape, color, dot, line, and style. (Such statements also 
include form - the expressed ideas and conveyed meanings behind the elements that give support to the 
communication of the visual statement (Seels, 1994). To effectively communicate with visuals, creators of web 
pages must consider the simplicity and clarity of the images, balance, harmony and organization of the text and 
images, aspects of framing, and emphasis color, texture, and space. (Thompson, 1994). 

Icons to serve as “reference points. ...for searching and retrieving information” (Ma, 1995, p. 33). Icons 
also can be used as signs. Signs are “...organized into systems of meanings or codes” (Muffoletto, 1994). Signs 
can also be influenced by social and cultural meanings. Ma (1994) studied the meanings of icons on the WWW 
pages of 150 United States libraries. A sample of icons Ma found included an icon depicting a web overlaid with a 
magnifying glass to depict Internet resources, book-shaped icons that represent on-line catalogs, periodicals, etc., and 
icons that portray a highway to catalog Internet resources. Semiotic analysis of icons indicated that familiar icons 
used to index traditional library resources were being used to index resources indicative to the resources on the 
WWW. 

Backgrounds and background images are important design components. Used appropriately, background 
images can enhance the foreground icons, graphics, and text. Many novice designers include these images to “add a 
theme.. .or because they fill up unused space” (Siegel, 1996, p. 131). 

Image maps (complex images that produce multiple hyperlinks) contain visual, navigational, and practical 
characteristics. Visual characteristics are clues that are recognized by the viewer and meet the viewer’s expectations 
pertinent to the information. Navigational characteristics include the viewer recognition and initiation of linked 
information being used to index resources indicative to the resources on the WWW. 

Practical characteristics include technical aspects such as image load time, background and special effects not 
cluttering the vision of the viewer, and whether graphics fit the target audience (Cochenour, Lee, and Wilkins, 
1995). Cochenour and others (1995) evaluated nine WWW sites that contained image maps for visual, navigational, 
and practical characteristics. Nine viewers evaluated the sites and “they seemed to judge the degree of appeal by how 
easily they could get to information 4 * (p. 170). The web sites receiving the highest ranks contained simple images 
and a well-ordered layout with clearly defined hot spots and a limited number of choices. Hypertext links are critical 
to web page design. These links are words or phrases that are underlined and colored to denote a link to more 
information or other web sites. The conventional use of color displays the “unvisited” links as blue and the “visited” 
links as purple. 

This study investigated the use of these design elements in software and printed materials that tell how to 
create web pages and in the web pages created by students and teachers in elementary, middle, and high schools as 
well as those in colleges and universities. 

Methodology 

For this research the web sites were categorized into two main groups, educational and noneducational. 
Then each of those larger categories were further subdivided. Figure 1 shows the organization of the educational web 
page site categories and Figure 2 shows the organization of the non-educational web page site categories. 

Figure L Educational Web Site Categories 









Elemenentary School 


— Student, 


Teacher, 


School 


Middle School 


-- Student, 


Teacher, 


School 


High School 


-- Student, 


Teacher, 


School 


College or University 


— Student, 


Teacher, 


School 




l 



n 



1 



141 



Figure 2. Non-Educational Web Site Categories 
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Development of Evaluation Instrument 

Considering there is no previous research about designing web pages from the perspective of visual literacy, 
the researchers began by examining the literature for checklists that related to the basic elements of visual 
statements, GUI design, hypermedia design, and publishing on the WWW (Curtiss, 1987; Lucas, 1991; Schwier, 
1991; Cates, 1992; Tolhurst, 1992; Wiggins, 1995). From these checklists two evaluation instruments 
were developed and refined based upon a six-point Likert-type scale. One instrument contained 21 questions for 
evaluating on-line and printed resource materials. Examples of the evaluation criteria in this instrument are: 

• Contains introductory information about the general WWW environment 

• Use of browsers 

• Explanations of image formats (tiff, pict, jpeg, gif, etc.) 

• Use of color 

• Links 

• Publishing on the WWW 

• Examples provided 

• Emerging trends 

A second evaluation instrument contained 57 questions that were embedded in the categories: design, 
graphics, text, and color. These categories were divided into aesthetics and functionality. These questions were used 
for evaluating K-12 schools, and college/university web sites. This instrument was also used to evaluate 
informational and cultural as well as personal pages. A sample of the evaluation criteria include: 

• Design — aesthetics 

• Design elements are aesthetically consistent (consistent headers, background, font sizes, etc.). 

• The page has appropriate white space. 

• The background contributes to the overall design. 

• Design — functionality 

• The design appears to be created for lay computer users not experts. 

• The message is clear. 

• The message is concise. 

• The site appears to be current. 

• A text only option is available when large graphics load slowly. 

• The links function properly. 

• Graphics load with reasonable speed. 
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• Graphics — aesthetics 

• The graphics do not distract the user from the main message of the page. 

• The graphics that are intended to be buttons look like buttons (easily recognizable as buttons) 

• The graphics that are not intended to be buttons do not look like buttons (you do not do false clicks hoping 
to go somewhere). 

• Graphics are easily interpreted. 

• Graphics - functionality 

• The site uses appropriate thumbnail graphics (user doesn’t need to use time and memory loading in memory 
intensive graphics). 

• Graphics enhance the message of the page. 

• Text — aesthetics 

• The properties of the text (font, style, size, color, pattern, etc.) are aesthetically appealing. 

• The color of the text is aesthetically compatible with other design elements on the page. 

• The layout of the text is aesthetically pleasing. 

• Text — functionality 

• The color of the text does not distract the user from the main message of the page. 

• The layout of the text does not distract the user from the main message of the page. 

• Color -- aesthetics 

• Color changes are used to convey a message (visited links, happy-sad, day-night, timed elapsed, etc.) 

• The design makes use of warm (red, orange, yellow) and cool (purple, blue, green) colors. 

• Color — functionality 

• The design of colors considers the age of the final user. 

• The color scheme is appropriate for color-blind users. 

• The background color is aesthetically pleasing. 

WWW Site Selection 

Sites were randomly selected from the web. Various descriptors, such as, creating web pages, HTML, web 
page design, and so on, were used in various search engines to illicit web sites containing resources about how to 
create web pages. An initial list of publishers who produce information about how to design and create web pages 
was constructed based upon a search of on-line, bookstore, and library materials. Over 100 electronic sites and 25 
printed manuals were evaluated. The web page sampling source for schools and colleges/universities was gathered 
from the Yahoo search engine using the education category. 

Twenty sites were randomly selected for each elementary, middle school, high school, college and university 
(n = 160). The K-12 web sites included both public and private United States schools. Higher education sites 
included public/private colleges and universities in the United States and four countries around the world. 

Independent Evaluators 

Four graduate students majoring in educational technology at a major mid-western university evaluated the 
commercial, informational, cultural, on-line and printed materials, and personal web sites. K-12 school and 
college/university web sites were evaluated by an associate and an assistant professor of educational technology. 
Both professors teach the use of the Internet and the WWW to graduate students. 

Results 

Initial data analysis of graphics, text, color, and design characteristics within the components of aesthetics 
and functionality indicates a wide range of quality in all the sites. This report contains qualitative descriptions that 
were used to help determine numerical scores. Quantitative analysis is in progress. 

Resource Materials 

Evaluation of the printed and on-line materials indicates a strong agreement in the use of how to create in 
HTML language and the technical aspects of using specific image formats (gif, tiff, jpeg, mpeg) for WWW 
publishing. Examples of these printed and on-line publications include: 

Fry, A. and Paul, D. (1995). How to publish on the internet. New York, NY: Warner Books 
Danesh, A. (1996). Teach yourself web page design. Indianapolis, IN: Sams.net Publishing 
Home page creator [On-line], www.angelfire.com/freepages/create.html 



Netamorphix [On-line]. http://trace.wisc.edu/TEXT/GUIDELNS/HTMLGIDE/htmlfull.html 
Lynch, P. (1995). Yale C/AIM WWW style manual [On-line], 
http ://info.med .yale.edu/caim/printinfo.html 

Niederst, J. & Freedman, E. (1996). Designing for the web: Getting started in a new medium. Sabastopol, 
CA : O’Riley 7 Associates, Inc.. 

Rogers, S. & Wise, A. (1996). Home page beautiful. Radiant Productions. [On-line]. 
http://click.com.av/click/v03/deconstruction/index.html 

Wilson, S. (1995). World wide web design guide: Learn to design professional web pages. Indianapolis, 
IN: Hayden Books. 

Although there is an abundance of resources for learning how to compose WWW pages from a technical 
standpoint, there is little agreement about the inclusion of elements of good screen design, appropriate size of 
graphics, use of icons for navigational purposes, and designing the screen as a portrait. That is, the message is the 
object and the portrait is the screen. Few resources in print and on-line, choose to address this type of information. 
In the cases that did, the authors discussed the use and aesthetics of images, backgrounds, text, graphics functionality 
and emerging trends. These authors usually provide the user with examples in the printed publication that correlate 
to examples in the on-line publications: 

Horton, W., Taylor, L., Ianacio, A. & Hoft, N. (1996). The web page design cookbook: All the 
ingredients you need to create 5 star web pages. New York, NY: John Wiley & Sons, Inc. 

Horton, W. (1991). Illustrating computer documentation. New York, NY: John Wiley & Sons, Inc. 
Siegel, D. (1996). Creating Killer Web Sites. Indianapolis, IN: Hayden Books. [On-line] 

http://www.killersites.com. 

Educational Sites 

Evaluation of school sites indicates a strong use of backgrounds that tend to distract from the overall 
message. School sites tend to use text rather than images to present information. They use text whose properties 
(color, font, style, size) are difficult to read because the background interferes. Also, they use text that is targeted for 
adult users rather than the age of students attending these schools. Examples of these sites include: 

Bethesda-Chevy Chase High School [On-line], www.mcps.kl2.md.us/schools/bcchs/ 

Chaffee Elementary School [On-line]. www.traveller.com/~lpearce/Chaffee/Chaffee.html 

Fall Brook Union High School [On-line]. sd.znet.com/~schester/FUHS/index.html 

Hardyston Township Elementary School [On-line], www.garden.net/users/hardyston/non-enhanced.html 

Longfellow Elementary School [On-line] www.nothinbut.net/~dhannah/xlongfel.html 

Orange Grove Elementary School [On-line] www. awod. com/ gallery/ rwav/ oge/ 

Quail Run Elementary School [On-line] www. sped, ukans. edu/ -scottk/ qr/ 

Examples of K-12 schools whose web sites contain good screen design, graphics that are aesthetically and 
functionally appropriate include: 

Brookfield Central High School [On-line], www.axisnet.net/~bchs/ 

Burke, Harry A. High School [On-line]. www.esul9.kl2.ne.us/burke/BHS.html 
Kyerene Akimel A- A1 Middle School [On-line]. 204. 17.16. 101/Akimel/ams.html 
Robinson, Andrew Elementary School [On-line], www.rockets.org/ 

A further analysis of school sites indicates that the functions of graphics are not always apparent. Graphics 
which were not intended to be buttons looked like buttons, and sometimes graphics which were buttons did not look 
like buttons, thus confusing the user. 

Whereas K-12 schools use web sites as a way of introducing themselves to the WWW and promoting their 
school community, colleges and universities use their web sites for a variety of purposes, i.e., new student 
application, on-line registration, promotion, on-line courses, announcements, faculty and student homepages. Data 
for colleges and universities indicates a strong agreement in the areas of good screen design, use of color, white 
space, and icons that are effective navigational tools. Examples include: 

Arizona State University [On-line], www.asu.edu/asuweb/ 

College of the Menominee Nation[On-line] www.menominee.com/ 
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Design functionality data regarding the updating of information, clues that reflect the information size, or 
provide a text only option when memory-intensive graphics are present indicates strong disagreement among all 
many K-12 schools and colleges and university web sites. However, many commercial, personal, and educational 
web pages contain indications of last update, informational size or provide a text option. 

The Santa Monica High School web site is a graphically sophisticated site 
(http://coke.physics.ucla.edu/laptag/SMHS.dir/). Information about the school ranges from a virtual tour, to 
schedules for lunch, sports, and classes. The home page is constructed with frames and a banner that have a 
professional appearance. A sound file loads auto with the home pages and this is very annoying. It takes along time 
to load and was distorted by the slow connection. The virtual tour was mostly done in text, which was a surprise. 
Although one would expect graphics, the text was informative and each site on the tour was described in paragraphs 
with appropriate links. Navigation is logical and easy to follow. 

The site give the impression that teams of students are responsible for various sections of it. For example, 
the sports section begins with a black background and a large image map that loads in a puzzling way. Small circles 
eventually appear around the logo and when the cursor is moved over the circles, labels pop up. When a label is 
clicked, it links to another page. Some users might find this confusing, but there is also a menu bar that is 
available, so people can use that. The menu bar remained present on all screens, so the user was not likely to get 
lost. 

The colors of the home page are blue and yellow. All pages have an advertisement at the top of the page 
indicated that the site is sponsored in part by different businesses. Overall, this site was a Spielberg-like in 
appearance in that is was very colorful an attractive. Although the graphics were slow to load, the users were given 
the choice to skip them and select a textual version. The downfall of this page is that typical users will need to have 
a lot of time to load the graphics and are therefore likely to skip the graphic version. This, of course, detracts from 
the visual potential of the WWW. 

The Andrew Robinson Elementary site (http://www.rockets.org/) represents a large, inner-city elementary 
school. This site received a “Florida’s Best” award form the Florida Communities Network. The site was created 
with the leadership of the three teachers and three students, representing a team effort in both content and appearance. 
The home page features a gif image of the school building and from there, users can take a virtual tour. The student 
involvement is evident in the graphic depictions and in the content included. Student drawings are featured 
prominently and the text is appropriately sized. Colors in the drawings are not always well coordinated, but they 
represent what the elementary students have chosen in their art work. The site gives a feeling of energy and 
enthusiasm. 

The majority of the educational sites reflect limited knowledge of design skills in terms of layout, 
aesthetics, interactivity, an general functionality. Much of the material is creative and useful, but the message is 
often lost in the quality of visual communication techniques that are used. 

Noneducational Sites 

In general, the noneducational sites reflect similar patterns of quality as the educaional sites, with the 
exception the commercial sites. The commercial sites range from poor to excellent, and it is apparent that some of 
the commercial sites have access to graphic designers and other professionals who have been trained in the 
construction of interactive computer-based materials that communicate. To use a sports analogy, anyone can figure 
out how to play softball, and those with interest can attempt to coach it, but there is a big difference between the 
winnings of an experienced coach and a novice who has no training. Likewise, the range of skill reflected in the 
WWW home pages ranges reflects all levels from beginners to professionals. 

Commercial Sites 

The commercial sites developed by professionals contained animations and sophisticated graphics which met 
the criteria for evaluation but they tended to target a narrow group of people. For example, product information sites 
can be very technical and difficult for the common user to understand. An example is the About Abbey Camera 
(http://abbeycamera.com/) web page. This site is targeted for general audiences but the navigational icons are not 
consistent and tend to mislead the user as to their use. 

Federal Express (http://fedex.com/), and Land *s End (http://www.landsend.com/) are examples of sites that 
target specific users. These sites contain icons and links that guide users who want to find shipping information, 
ship parcels, or order items. The icons are understandable and link to pertinent information about shipping parcels 
and other aspects of the each companies’ business. 
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Some of the web pages are well designed and meet the needs of the general user. Examples of such sites are 
those for Kinko's (http://www.kinkos.com/main), JC Penney’ s (http://www.jcpenney.com/), and the Ben and Jerry's 
Ice Cream (http://www.benjerry.com). At these sites, the text and images are matched for meaning, consistently 
used, and appropriate for general adult users. The Idea Enterprises site (http://www.ieaudio.com/webpage/html) has 
some interesting features but it also employs some flashy features that detract from the pages as well. The Kinko's , 
Ben and Jerry 's Ice Cream , and Idea Enterprises sites are discussed in more detail below. 

Graphics at the Kinko's site are placed in an informal balance to form a dynamic design The informal 
balance creates a path that directs the visual path of the viewer (Amheim, 1974). Each succeeding graphic object 
leads the viewer to the next object. The use of informal balance requires considerable white space on a page. Near 
the center of the page on the left side there is a large area of white space that forms a void area leaving the viewer to 
wonder what should be there. The white background does provide sharp, clear contrast for the graphics. Short 
blocks of text placed next to the graphic provide a preview of the content on the page that can be accessed by that 
graphic (Horton, 1991). Red type matching the red in the logo is used to call attention to two items on the page. 
Lower case type is used to indicate message of equality with the reader on a conversational level. Color balance in 
the page is achieved by the use of small areas of warm bright colors in the graphics of setting larger areas of the 
graphic's cool colors. Additionally, the logo colors, blue and red, along with a small area of orange are repeated r in 
the computer screen of the last graphic of the page. 

The consistently up-to-date web page of Ben & Jerry's Ice Cream Company changes as often as daily to 
acknowledge the holidays, commemorations, and celebrations of our culturally diverse nation. Even though the 
content changes, the banner consisting of the company trademark and signature line, provides the viewer with a 
familiar greeting, much like that of a 

newsletter. The banner, in a smaller size is repeated on each of the succeeding pages. Unlike most commercial 
pages, the introduction features a cultural event or events rather than company information. Each event is illustrated 
with a tasteful graphic or picture. Short blocks of bold or color accented text accompanies each graphic. These 
design features are closely aligned with the recommendations of Horton, Taylor, Ignacio and Hoft (1996) for the 
introductory area of commercial home pages. Effective use of the full color spectrum is accomplished through the 
thin graphic (horizontal rule or swashes) spanning the width of the page and separating sections of the page. 

During the Holiday Season this graphic was a string of multicolored lights, emphasizing the message of the 
opening paragraph and graphic. Rules are used to coordinate page colors and separate sections of the page (Horton, et 
al., 1996). In January, the event of focus was Martin Luther King Day. Miniature drawings of children, depicting 
various cultures were placed holding hands to form the rule graphic. The same rule graphic through out the page to 
create subject separation and provide color coordination for the various sections headlined with different colors. The 
overall design of the page closely parallels the design of a printed newsletter. Accommodating the large amount of 
information on this page, is possible through the use of creative, well planned, artistic design. While sections of the 
page promote the companies ice cream and frozen products, the pronounced message is one of a culturally aware, 
socially conscious company. 

Idea Enterprises' home page opens with a large, bright purple and black logo on a textured turquoise 
background. The following text is centered just below the logo, " idea enterprises that's advertising , Offering Web 
page development and renovation, Web audio content and radio ad campaigns for your business ." Following those 
words is a listing of links to subsequent pages advertising the services available, including one for web page design. 
Phone and e-mail information is displayed below the table. 

At first glance this home page captures the viewers attention. However, a closer look at the site reveals a 
lack of navigational buttons, inconsistent banners on linked pages, and inconsistent use of animated graphics. In her 
book Designing for the Web , Jennifer Niederst emphasizes the need for navigational tools on each page of the site by 
reminding the designer that a user can enter a site at any point (Niederst, 1996). There is no guarantee the person 
viewing the site will arrive by way of the home page and could possibly never arrive at the home page without 
navigational buttons. There are no navigational buttons on the " web page design " page to take the user back to the 
home page or to other pages in the site. Additionally, the banner on this page reads " Simply put... Look around 
our web site. ..seeing." In that line the word "seeing" is in purple type, an indication that it is probably a link, but 
when clicked on, nothing happens. After clicking on the words in purple type scattered through out the page, the 
user will likely conclude these are not links but are possibly a part the message. 

All graphics on this page are centered horizontally and animated. The first is a yellow smiley face that 
appears to be talking. Just below the talking smiley face is a flashing rectangle with a background that changes 
from blue to orange and flashes the words "Click, ”, "Here," "Do It," "Now" in bright yellow type. At the vertical 
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center of the page is an animated picture of a human eye looking from side to side. Next, two short lines of type 
separate the glancing eye from a small rectangular graphic containing a dancing flame on a black background. The 
final graphic on the page is an e-mail icon miming an invitation to contact the company via e-mail. 

Animation does draw attention and stimulate interest but should be used to communicate not to decorate 
(Horton, 1991). The preponderance of animated graphics on this page draws attention to the extent of overpowering 
the advertising message. 

Informational Homes Pages 

It seems that a major difference in the amount of text or graphics used on a page was related to the 
availability of resources to design web pages. This pattern is reflective of the resources available to larger 
communities which have sites usually designed and supported by professional designers. Whereas, the more rural 
sites are maintained by a technical support staff. For example, web pages for major metropolitan areas use more 
graphics, animation, and color throughout the site than sites for rural communities. The more rural or remote a 
community , the greater the use of text to convey the message. Some rural communities that use a number of 
graphic elements on their site, are a regional grouping of several communities that centralized their resources 
(CedarNet; Ames Electronic Village). Also, a majority of military sites initially used graphics on the home page, 
but revert to text on linked pages of information. This pattern possibly follows the military style of training which 
is based upon getting and individual attention and then presenting the information. 

Informational web pages for museums and libraries, which also included zoos and major park associations, 
and special interest also follow this same pattern. A majority of these organizations have resources for design, 
which assist them in marketing themselves with graphical themes, and transferable to the design of web page. 

The structure of the CedarNet (http://www.cedamet.org/navigate.html) home page places a focus on formal 
balance between the main graphic and the text in a horizontal list, this provides ample white space around both the 
graphic and the text. Central tot he home page is the focus on the main graphic, aesthetically pleasing in its use of 
shape, color and realistic images. The graphic is circular in shape with realistic images or pictures of buildings, 
appropriately labeled, representative of the varies functional parts of a local community, this graphic gives the home 
page a friendly, Norman Rockwell hometown message to the user. Each building in the graphic is used as a graphic 
button, linking the user to subheadings of categorized information available in this community. 

The main graphic changes with the seasons of the year. The key graphics and labels remain the same, but 
the graphic changes to represent the them of the four seasons throughout the year. The winter graphic makes use of 
winter scenes and activities, snow, and the use of cool colors (blue, green, purple) and white. Autumn is represented 
in this graphic by the use of more warm colors (red, orange, and yellow) and appropriate activities. For spring and 
summer the main color is green. 

In terms of functionality, the graphic elements and the text list are linked to provide navigational options to 
the user accessing information. Linked pages are appropriately titled and most have the corresponding building 
graphic on the header of the page. But the graphics do not carry over the seasonal as presented on the home pages. 
Also, animation is used in the header graphic on only two of the linked pages: the neighborhood page, and the 
recreation and tourism page. 

The only major inconsistency of this site is the use of background colors on linked pages. The color and 
texture vary from linked page to linked page. The background colors range from green, yellow, white, gray, and a 
pastel spectrum of colors. Also, the layouts on the linked pages vary from horizontal and vertical lists. 

This inconsistency contradicts the CedarNet Web Site Design an Development Committee statement of 
purpose for “maintaining a cohesive, consistent, and attractive web presence in the main structure areas of CedarNet.** 
CedarNet defines itself as an electronic community dedicated to providing quality community information in 
Northeast Iowa, representing the communities of Waterloo an Cedar Falls. 

The principal design of the Whidbey Island Naval Air Station (http://www.naswi.navy.mil/) web page uses 
a formal balance between the graphic elements. The design places the logo/graphic and title at the top half, and the 
graphical navigation buttons in the lower portion of the page. This creates a portrait view of the page, designed to 
be seen all at once. The layout of the page makes use of appropriate white space around the text and graphic 
elements, which creates an inverted pyramid flow of the viewer eye towards the navigational buttons. 

Information on the home page is concise and labeled to leads the use directly to additional information. The 
title, as a graphic element, is four lines of large centered italic text, arranged in a complementary shape to the 
triangle graphic next to it. Graphical lines, lines a with small graphics element, are used to separate the title and 
navigational areas of the page. The graphic element depicting with these lines shows a jet and exhaust trail moving 
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right and left across the page. These lines are also used on linked pages as a element of the page header to separate 
the title from the page information, or to separate groups of information. Additional thin graphical lines, using a 
pair of “peering” eyes, are used at the bottom of the page as a means of separating and calling attention to important 
information and warnings. 

The focus of the navigational area of the page are the twelve three-dimensional blue-colored buttons. 
Layout is three buttons across the page and four down. The button labels are in a yellow italic text, making good 
use of complementary primary color . Centered text above the buttons indicated a special message in bold, and a 
corresponding list of button labels is below the buttons. 

The shape of the buttons crate a problem of inconsistency with other elements of the page and site, the 
home page navigational buttons are the only three-dimensional graphic on the page. This causes the other graphic 
elements to have a flatten appearance. Also, the buttons on linked pages changes shape from the rectangles to blue 
spheres. A consistent use of shapes and lines would add to the overall appearance of the page. 

Blue is used as the primary color for the graphics and text. Additional uses of color in the graphics include 
shades of cool colors - blue and green, and complementary primary colors of red and yellow. Text used on the 
graphic is white on a blue background. Visited sites on the page, the text listing and the outside edge of the button, 
are highlight in a reddish-brown color. This color may cause problems for colorblind users. 

Text on the home page is used at an appropriate size and style to read easily. But, the text on the linked 
pages appears smaller and more difficult to read due to added white space or changes in the color of the background 
(white). An increase in the text size on these pages would add to the readability to the site. 

The National Center for Fathering web page (http://www.fathers.com/) presents a formal balance of the 
graphic elements - the logo/title center top followed with two column listing of subtopics, creating a portrait view 
of the page. This site shows the touch of a professional design team in it’s overall appearance and use of graphic 
elements. The page’s layout makes use of appropriate white space which brings a realistic view to the “bulletin 
board” concept of the page design. 

Information on the page is concise and appropriately labeled, and the use of a color with the linked-text is 
consistent with the logo. The uses of realistic graphics, pushpins and strips of yellow legal pad paper, for buttons 
and masking tape for the navigational bar, gives the page depth. The graphic elements used are easy for the user to 
identify and associate with, but it is difficult time to picture how to put pushpins into a white marker board. A little 
texture to the background would give the realistic appearance of a white bulletin board. 

The design concept is carried through to the header graphic, a strip of masking tape, on linked pages. The 
home page is the only location on the site where the same header graphic is not used. The pushpin and paper theme 
is carried through on the linked pages for listings of subtopic information. The use of masking tape for the 
navigation bars on each, the home and linked pages, is consistent in its use 

Linked pages have a consistent pattern to the layout of the page - the logo in the upper left, graphics related 
to the topic to the right, followed with the masking tape header. Information below the header on these pages varied 
in amount and layout, but this was where the site showed it inconsistency. Some large amounts of information was 
listed with the pushpin/paper graphics, others the user had to scroll down. 

Conclusions 

With the growing cost associated with on-line access, the technical hardware and software needed to view 
web pages, the purpose of design has greater value for an informational sites. Understanding the relationship 
between available resources and page design is valuable in that it may helps explain that there is more to web pages 
than the technology that enables us to see them. For if a web page, and the available information on the WWW, is 
to be used to its fullest potential, the appropriate use of design of that page has value. For the purpose of design is 
to make sure that the information presented in a useful and clear manner has value. 

Preliminary data analysis indicates that the majority of the web pages use a simple design, with the 
standard blue and purple colors used for unvisited and visited sites. Few web pages contained clues that would 
indicate the size of the files in the site. The majority of web page backgrounds used distracting patterns and/or color. 
In many cases the designers seemed to forget to maintain message of the page. There was a tendency in most of the 
personal pages not to indicate the path the user is following. The user should be able to tell where he or she came 
from, how to go back and find his or her way around. Many sites did not indicate whether the site had been updated. 
This may lead the user to loose interest in the site. In many cases, graphics that looked like buttons were not. 

The use of thumbnail graphics was minimal. 
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Recommendations 

The next phase of this study is to perform an in-depth data analysis. Following that, a formal evaluation of 
the data should be completed. From a preliminary analysis, the following considerations merit investigation: 

• Should the elements of the visual arts, dot, line, color, text that have been standard for decades apply to the 
medium of on-line publications? 

• Which literary publication conventions should or should not be applied to the unique environment of web 
pages? 

• In what ways do the questions of interactivity, connection and interconnections, user control, and expeditious 
information gathering contribute to new publishing conventions? 

• How does the relationship of not knowing how a publication is structured impact the ability to communicate? 

Most certianly there is rapid growth to be expected in the WWW environment an guidelines are needed to 
help people develop the most effective pages for communication and for instructional purposes. Resources that 
address the technical aspects of creating WWW pages fall short of those needs, yet consumers may not realize the 
need for training in this area. Educators who are pressed to learn new technical skills yet have increased 
responsibilites which lead to extremely short amounts of time available, may find that partnerships with commercial 
enterprises will be particularly helpful as schools move into more WWW communication. 
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The use of cooperative learning methods in distance education should provide an extremely pertinent instructional 
strategy, which should be particularly effective at "overcoming the distance" "maximizing interactions", and providing social 
support. In the information gathered from surveys, interviews and a review of the literature, these are the most frequently 
occuring constraints to the distance learner's motivation and achievement. They are also key contributors to the high drop-out 
rate in many distance education courses. By actively creating a learning environment that advocates peer interactions, "social 
support", and "promotive interpersonal communications ", cooperative learning models can help attain that sense of a learning 
community which is frequently lacking in distance education experiences. A review of various cooperative learning models 
and the types of learning activities each supports are investigated for practical applications in a distance education setting. 



In order to promote successful distance education techniques, the University of Minnesota developed a training 
program to prepare faculty to instruct via their ITV delivery system. The need for specific areas of faculty development were 
assessed through a survey of current practitioners, as well as a content analysis of existing training materials from other 
institutions. This resulted in the creation of an information matrix that was categorized into distinct topics and associated 
sub-classifications of the prominent training issues concerning distance education. The matrix categories were developed by 
using a "Delphi technique", a review of the literature, and interviews conducted with identified experts in the field. Individual 
matrixes were developed for each institution and compiled into an overview matrix which demonstrated the overall frequency 
of key elements based on survey responses (Kochery, 1995; see Tables 1&2). 

The matrix revealed that the most frequendy occurring sub-categories of faculty training and content development 
needs were: "Maximizing Interactions and Feedback" and "Developing TV Lectures ". These two instructional "design" issues 
both emphasize that distance education requires more attention to strategies that promote interactions: questioning techniques, 
discussions, active learning. The matrix results concur with the current emphasis found in the literature on the need for 
"maximizing interactions" in a distance education environment (Cyrs and Smith, 1990; Wagner, 1993; Willis, 1993). There 
is prevailing evidence that a major concern of many practitioners is to identify design and delivery strategies that prevent a 
potential barrier to this form of instruction by "overcoming the distance". The instructional methods that are most frequently 
cited all involve plans which stimulate active learning, by using procedures that purposefully increase a dynamic student 
participation with their own learning process. 

There are many special concerns that influence the increased importance of interactions for distance learning 
situations. One of these is the fact that many distance learners feel isolated and alone in the essential social transactions that 
are manifested in most educational settings. These learners not only feel separated from interactive feedback with the 
instructor, but they also feel separated from meaningful exchanges and the sense of a shared experience with their fellow 
students. The sense of isolation and the lack of social support have frequently been cited as primary reasons for the 
disturbingly high drop-out rate during many distance education courses. Distance learners feel the lack of emotional intimacy 
that exists for personalized constructive feedback and they are often frustrated by the "unapproachable" physical condition 
which detaches them from the instructor. "It is the physical separation that leads to a psychological and communications 
gap, a space of potential misunderstanding between the inputs of the instructor and those of the learner, and this is the 
transactional distance." (Moore, 1992) 

Many strategies have been suggested to "maximize interactions", however most are still dependent on a teacher- 
centered design and delivery of instruction. The true "distance" of the communication problem remains the same: the teacher 
is still viewed as the central figure who controls and transmits knowledge to a receiving audience. Effective learning is 
dependent on active negotiations and explorations of "meaning", not a passive individual experience based on mere exposition, 
reception, and retention. Distance learners frequently feel isolated and removed from their educational experience which is in 
direct opposition to one of the primary tenets of many educators that learning is essentially a "social process" (Brown & 
Duguid, 1993). In essence it is considered to be a fundamental part of the learning process that people actively construct and 
reconstruct their worlds during interactions which are facilitated by social affiliation. 
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Most learning in most settings is a communal activity, a sharing of culture. It is not just that the child 
must make his knowledge his own, but that he must make it his own in the community of those who share 
his sense of belonging to a culture. 

(Bruner, 1983) 

Rationale for Cooperative Learning 

All forms of distance education, regardless of the method of delivery, need to utilize one of the most direct means of 
increasing student involvement, participation, and interaction with their own learning environment - cooperative learning. 
Indeed, one of the primary effects of cooperative learning is an increased sense of social support and an emphasis on positive 
personal interactions. This is manifested through instruction that emphasizes the benefits of helpful peer interactions and 
positive group experiences. Formal cooperative learning has socially interactive qualities to help overcome the "transactional 
distance" and feelings of alienation, while positively effecting student performance and satisfaction. While maintaining and 
eliciting interactions in any educational setting are important, there are many special concerns that influence the importance of 
creating increased opportunities for interactions in distance learning situations: 

Results of perceptions indicate that distance education students experience less 
involvement, less ability to ask questions and less overall enjoyment. In this 
situation, interaction may need to be more structured to compensate for the 
limitations of the particular communication system. Those in the traditional 
setting experienced more comfort, had greater opportunity to ask questions, and 
felt they were more involved in the instructional process than those in the 
distance education group. (Ritchie 

and Newby, 1989) 

Cooperative learning is based on grouping students together to help each other attain common learning goals. The 
success of the group process is dependent on "encouraging students to discuss, debate, disagree and ultimately to teach one 
another" (Slavin, 1991). Cooperation becomes a function of the combined efforts of peers working together in such a manner 
that the actions of neither party actively opposes the productive efforts of the other party, rather they support, promote and 
share actions. Learning becomes viewed as a dynamic dialogue between the internal cognitions of each student and the 
sharing of those perceptions with peers. In a meta-analysis of the existing research (Johnson, Johnson, and Maruyama, 1983) 
found that cooperative learning produces a positive effect on student’s motivation, self-esteem, and academic achievement: 
Learners tend to generate higher level reasoning strategies, a greater diversity of 
ideas and procedures, more critical thinking, and more creative responses when 
they are actively learning in cooperative groups than when they learn 
individualistically or competitively. 

More recently, Johnson and Johnson (1991) found that in addition to the above results, cooperative learning 
strategies also promote: higher retention rates, better attitudes towards instruction, increased on-task behavior, and greater 
collaborative skills. Slavin (1991) reports that cooperative learning increases academic achievement at a variety of grade 
levels and across diverse subject matter areas. He has also found evidence that cooperative learning improves intergroup 
relations, it improves the relations between handicapped "mainstream" students and their "normal-progressed" peers, and it 
improves most student's self-esteem. He maintains that: "Cooperative learning methods generally work equally well for all 
types of students". There is also evidence, that both low achievers and high achievers benefit from the "exchange of 
elaborated explanations" (Webb, 1985) which apparently stimulates cognitive growth and restructuring. 

Many studies have shown that cooperative learning with CAI positively effects student motivation and learning 
outcomes (Beckwith, 1993; Dalton, Hannifan, & Hooper ,1989; Johnson, Johnson and Stanne, 1985). Other studies have 
demonstrated the benefits of cooperative learning and distance education transmitted across computer networks (Davies, 1988; 
O’Malley and Scanlon, 1990; Riel, 1990). Investigations have also been made which indicate that group work and more 
frequent interactions will have a positive effect on televised instruction (Ritchie and Newby, 1989; Klein and Pridemore, 
1992). 

These findings would appear to be particularly applicable to distance education concerns, where the normal 
opportunities for interactions and socializations that can be found in a traditional classroom are confounded by physical 
distance. The very nature of the feedback becomes an artificial representation of a "live" social interaction, because there are 
always some major channels of communication such as touch, body movements and interpersonal distance which are absent. 
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Electronic communications systems often create an interpersonal distance, a barrier in the form of a"public distance zone”, 
which lacks personal intimacy and connotes feelings of differential social status and a formal communication atmosphere. 

Cooperative Learning Models 

There are many methods of cooperative learning, but they are all predicated on creating learning environments where 
students exchange their perspectives, promote mutual individual development, and maximize the achievement of the group . 
Robert Slavin's Student Team Learning (STL), emphasizes “group rewards” to motivate student performance: "... the 

students' tasks are not to do something as a team, but to learn something as a team.” (Slavin,1991). The central element of 
the STL is the "Team Rewards” which are based on the group achievement of specified criterion. A second element is 
"Individual Accountability”: the responsibility of each team member to learn, and the group's responsibility for explaining 
concepts to one another and thus assuring the individual development that will ultimately benefit the overall team 
achievement. The last major element of the STL is "Equal Opportunity for Success”, which means that each student 
contributes to their team by constantly improving their individual performance. 

The "Jigsaw" method by Elliot Aronson (1978) depends on academic material being broken down into separate 
sections. Student groups are formed and each member of a given "home” group is given the responsibility of a specific topic 
of the course content or project. Each member of these "home” groups then disbands to meet with students from different 
teams, who have been assigned to study the same content section or investigate a particular task, to form an "expert group”. 
Within these "expert groups” information is researched, shared and refined in order to arrive at an informed consensus 
concerning their assigned subject matter area or activity. The "expert groups" then disperse and return to their "regular" 
teams, where each "expert member” teaches the other group members about their content section. The only way for the 
students to learn about the material outside their assigned section is to depend on the knowledge and the instructional 
resources of their teammates. 

Sharan and Sharan (1980) have developed a Group Investigation method(GI) which is based on cooperative inquiry 
and classroom collaboration. Students choose their own groups, select a sub-topic from the unit of classroom study, divide 
this sub-topic into individual assignments, and then present a group report to the entire class. Emphasis is placed on the 
interdependence of the larger classroom group to these smaller investigation groups, and evaluation occurs at many levels. 
The team presentations are usually evaluated by the class and the instructor, while an individual’s contributions to the group’s 
performance are evaluated both by the instructor and the other group members. 

The Johnson and Johnson model "Learning Together" (1975) focuses on interactive social processes and team 
building activities which are predicated on developing important interpersonal social skills (leadership, trust-building, 
constructive communications, conflict management). Students are assigned to completely heterogeneous groups based on 
diverse ability, gender, race, and cultures. They also consider the analysis and evaluation of the actual "group process" to be 
an integral component of the learning experience: "Such processing enables learning groups to focus on group maintenance, 
facilitates the learning of social skills, ensures that members receive feedback on their participation, and reminds students to 
practice the small group skills required to work cooperatively." (Johnson and Johnson, 1991). The Johnson and Johnson 
cooperative learning model is predicated on structuring the learning experience to promote collaborative activities by 
implementing these following five basic elements: 

1 . Positive Interdependence : Mutual goals and shared rewards linked together in such a way that one member 

of the group cannot succeed unless the other members of the group succeed 

2. Face to Face Promotive Interaction : Establishing the behaviors of helping, assisting, encouraging and 

supporting each other's actions 

3. Individual Accountability : Each individual student’s performance is assessed and is based on the extent to 

which they have mastered specific materials and provided support and assistance to the group 

4. Interpersonal and Small Group Skills : Developing the social and communication skills of leadership, 

decision-making, trust-building, and conflict-management which support a group problem solving 

attitude 

5. Group Processing : Dynamic analysis and improvement of group operations, communications and 

performance 

It would appear that the implementation of structured method of cooperative learning would greatly benefit the 
achievement and the affect of distance learners. By actively creating a learning environment that advocates cognitive 
interactions with peers, stimulates "social support", and instigates "promotive interpersonal communications", cooperative 
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learning models can help attain that sense of a learning community which is frequently lacking in distance education 
experiences. 

Cooperative Learning Techniques 

These cooperative learning models supply instructional strategies which should positively effect the distance learners. 
"Turn to Your Partner" or "Bookends" involves breaking down instructional material into manageable thematic "chunks" and 
having learner dyads discuss these key question or review points. This presentation method aids in the pacing, processing, 
and retention of the content material. Some examples of using this method are: pose a direct discussion question or problem 
which asks for the analysis or synthesis of information; have the learning pairs generate a question or problem for class 
conversation; have the students elaborate on existing content and relate it to previous material, or have them reflect and react 
to a theory or concept. Have the learners work in pairs to process the information through some form of dialogue and follow 
this forum or methodology: 

1 . Each student independendy formulates his or her answer 

2. Students share their answer with their partner 

3. Students listen carefully to their partner’s answer 

4. Pairs create a new answer based on associating, building upon, and 
synthesizing each others thoughts 

After the pairs have processed the information, randomly call on any student in any group to provide their group 
answer. Asking questions in this manner helps to assure individual accountability: that the pairs have shared and explained 
their perspectives, that one student has not dominated, or that one member has not been a "free -rider" on the other’s efforts. 
While this is more conducive to a ’’live" synchronous learning experience - or in an ITV classroom separated only by location 
- it can also work with asynchronous learners who are separated by space and time. Assign distance learners to "study pairs" 
and have them communicate through email, conferencing software, or telephone - their individual responses can be shared 
synchronously or asynchronously and easily stored or documented. One benefit to this form of communication is that all 
contributions to the group process are actually recorded and can be monitored by the instructor to promote individual 
accountability. These same forms of communication can also be used to elicit "live" dynamic responses from the pairs or 
they can be assigned to document and "hand-in" their exchange of perspectives for assessment. This form of interaction may 
not be as dynamic as ’’live" discussion, but it accommodates to the needs of a distant learner, while still providing a social 
exchange of ideas and a forum to arrive at a consensus. 

Slavin's STL and the Sharans' GI models provide excellent methods for building cohesiveness and a sense of group 
identity with asynchronous learners and at remote sites. Team building exercises should be employed that encourage unity, 
which can be easily started by having the groups create names and mottos. It is important that the teams strive to define an 
agreeable set of group goals that provide a focused sense of purpose and instill self motivation. An initial group task should 
be a collaborative effort to create a mission statement that helps elaborate on the group’s intentions and expectations of 
performance: 



1. Define Mutual Goals: merely achieve passing grade; increase knowledge of 

subject matter; gain skill competency ; create a publishable research paper 

2. Group Reward Structure: group success if project receives a high grade; 

celebration/party if group achieves a specific group GPA; 

3. Set Standards of Achievement: all members get a B or better; all members 

contribute and participate; avoid conflict; provide constructive feedback 

4. Define Problems and Solutions: what are the group’s tasks; methods for 

achievement; individual roles and responsibilities 

The "team" concept and external group competition should be used as a metaphor to stimulate academic interest and 
esprit de corps , while the structure of the course should provide a focus on group homogeneity and achieving some agreed 
upon academic standard. However, a mild -mannered "gaming" format along the lines of the "College Bowl", "Jeopardy" or 
"The Hollywood Squares" can help build a sense of solidarity by grouping remote learners, while also providing a less- 
threatening academic activity that invites participation. Meanwhile, a more literal example of implementing Robert Slavin’s 
Student Team Learning (STL) model might be as follows: 




154 




• "Team Rewards": all team members must achieve above 80%; complete 
assigned tasks of the research project 

• "Individual Accountability": structured by giving a group grade based on an 
average of the individual members scores; peer evaluations 

• "Equal Opportunity for Success": improved performance by individual group 
members on papers/tests; group participation 

Teams can also be effectively engaged in role-playing simulations, where they must adopt, represent and advocate a 
certain perspective. At Pennsylvania State University, students participating in Project Vision courses - which emphasizes 
the use of communication technologies to promote asynchronous, anytime, anywhere instruction - are asked to simulate a 
public debate on the efficacy of a particular spot for use as a nuclear waste site. The students are assigned to groups and asked 
to characterize one of the factions having various interests or responsibilities for the waste site: Department of Environmental 
Protection, The Real Estate Coalition, Chamber of Commerce, Concerned Citizens Environmental Coalition, City Council. 
Each student group must research the concerns of their adopted "special interest group" and role play those viewpoints in a 
"virtual" on-line town meeting that is transmitted through conferencing software or a compressed video format. 

The Sharon's Group Investigation model provides an excellent procedure for assigning group presentations and an 
opportunity for peer teaching. Groups can be formed based on shared areas of interest, physical location (remote sites), or 
created by assigning asynchronous learners to "virtual groups" that will collaborate via various forms of communication. For 
typical ITV courses where the sessions are synchronous but separated by a distance, the student groups can share their 
presentation with remote sites by transmitting over compressed video. These "real time" presentations can use various media 
to illustrate concepts (video, computer graphics, presentation software), or they can include other modes of expression such as 
staging a play or a panel debate. For asynchronous learners separated by time as well as place, these group reports can be 
attached through email, posted as web pages, or produced as short videos. Similarly, fellow classmates can engage the 
presenters with questions or comments about the reports thorough email or by creating "virtual" hours with the presenters via 
phone or software conferences. The groups are responsible for educating their fellow students, while the remaining class 
audience must also be held accountable for processing the information content that is delivered. It is critical that evaluation 
and assessment be conducted on a number of levels: group project, individual contributions, knowledge 

acquisition/participation by whole class. 

Aronson's "Jigsaw" method creates discrete subject matter responsibilities and a sense of "positive interdependence" 
between individuals and their groups . This model lends itself well to case-method or problem-based learning situations, 
where the problems, tasks and subject matter can be readily divided-up. The "home" and "expert" groups can be purposely 
structured to expose participants to a mix of conditions and interactions with other learners: sometimes they may be separated 
by distance or time, sometimes they may be in a synchronous or asynchronous condition. Once again, it is necessary to 
provide various means of communication dependent on the nature and condition of the audience . The Aronson model 
simulates one real life benefit of groups, which is an increased productivity and efficiency through a division of labor. 
However, it is crucial that the final product is not merely a series of independent efforts and tasks which are loosely tied 
together. Individuals need to develop and share expertise with their groups and the remaining group members need to process 
and assimilate the information provided by individual members. It may be beneficial to have each group member attempt to 
create a "draft" of the final paper or project in order to promote both personal comprehension and the process of integrating the 
various pieces of information. 

The Johnsons' "Learning Together" (LT) model is extremely useful for addressing some of the issues and problems 
which normally arise when employing a cooperative learning method. Cooperative learning is more than merely placing 
students into small groups and expecting positive interactions. The five basic elements of the LT model provide a structure 
that promotes the creation of a learning environment which is conducive to cooperative behaviors. In order to have successful 
interactions it is necessary that learning groups are placed in a setting based on: positive interdependence, personal 
accountability, interpersonal skills, a shared sense of responsibility for each other’s and the group's achievement, and an 
awareness of evaluating the group processes (Johnson and Johnson, 1991). The various methods that can be used to 
implement the five basic elements provide some clear guidelines for valued behaviors. The Johnsons believe that is critical 
to create the'proper conditions that allow cooperative learning to be effective and productive. 

There are many barriers and many factors which can inhibit group performance and they have been tied together into 
a construct called "group effects". The framework provided by the cooperative elements of the LT model help to purposely 
structure activities and create lesson plans which encourage appropriate group behaviors. They provide a mechanism to avoid 
some of the common barriers to successful group performance: reticent or dominating participants, "free riding" on the efforts 
of others, social loafing, social status, task involvement, self-oriented needs, group think, and destructive conflict. LT 
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assists by providing a context for practicing individual roles and duties that are dependent on appropriate behaviors that will 
facilitate the group process: "checker”, "recorder", ’’facilitator” (see Table 3). Therefore, it is implicit within the Johnson & 
Johnson model that the development of interpersonal communication skills and group work become fundamental goals of the 
instruction. In order for cooperative learning to succeed, it must be integrated into the coursework and given a percentage of 
the tasks, practice, feedback and evaluation normally devoted to the subject matter content. 

Conclusion 

Because the various group effects and inhibitions can cause negative group functioning, cooperative learning will 
usually not be effective without structured strategies and training. Just as there must be changes to the focus and structure of 
learning activities, cooperative learning also emphasizes changes in assessment methods. Learners must learn how to give 
and receive peer evaluations in a positive manner. Furthermore they must also become aware of their own metacognitive 
processes allowing them to provide accurate self-evaluation. Collaborative instructional strategies require changes in the 
professor’s attitudes and behaviors as well as changes in students’ attitudes and behaviors. For example, professors must learn 
to share ownership of classroom activities with students and make the transition from "the sage on the stage” to the "guide on 
the side”. Likewise, students must take on more responsibility for their own learning and strive to become active 
participants with the educational process, not merely passive recepients. 

However, it is frequently the situation that both faculty and students are asked to work collaboratively and yet are not 
given training and support to help them acquire group communications and group problem-solving skills. Dedicated 
instruction needs to be provided that allows an opportunity for the following skills to be learned and developed: systematic 
problem solving, deferred judgment, checking for misunderstandings by paraphrasing, acceptance of conflict and conflict 
resolution, leadership, group management practices, understanding the stages of group progress (e.g., “forming, storming, 
norming, performing”), and using role differentiation. The study and practice of collaborative skills has to become 
incorporated into ongoing learning activities and interwoven throughout the curriculum as well as class assignments. The 
development of coop behaviors is a long-term process that cannot be attained through short-term interventions. These skills 
are also very difficult to advance from a distance, therefore instructors need to create methods that "check-in" and monitor the 
evolution of the group process. 

Of course the methods of implementing cooperative learning techniques are going to be dependent on numerous 
factors created by the circumstances and context of the instruction. What are the fundamental educational goals? Who is the 
primary audience? Where does instruction need to be conducted? How does it need to be delivered? In a situation that requires 
distance education, the nature and environment of both the audience and the content material will be critical factors in 
determining the appropriate methods of delivery which will help ascertain the cooperative learning methods that will be most 
effective. The various forms of electronic delivery ( FAX, ITV, Internet, computer and phone conferencing, email) each 
facilitate different modes of expression and communication. Indeed, attempting to merge the various capabilities and nuances 
of these new technologies with cooperative learning strategies and learner needs is a challenging proposition. Not to 
mention, that due to the varying degree of learner competency within these forms of communication, additional instructional 
time must be devoted not only to the ’’mechanics" of using these technologies, but also to providing examples of appropriate 
etiquette within these discourse formats: competent and effective email correspondence, the structure of an efficient computer 
conference (role of a mediator, equal participation, topics, categories of responses). However, it is also important not to 
overlook the need for some basic strategies to help overcome the "transactional distance", such as the social facilitation offered 
by attempting to arrange ’’live - face-to-face" interactions. 

The use of cooperative learning methods in distance education should provide an extremely pertinent instructional 
strategy, which would be particularly effective at "overcoming the distance” and "maximizing interactions". The new 
communication technologies enable educators to expand cooperative efforts between remote sites, diverse cultures, and 
"distant" expertise. The variety of electronic communications will now support more access to a range of perspectives and 
insights - providing the potential to build a global "web” of contacts and interfaces. These expanded modes of access and 
communications become an advantageous resource for all forms of distance education and they depend on a system of 
collaboration. Therefore, it would only seem practical, necessary and relevant to merge the instructional methods of 
cooperative learning with the new "enabling technologies" of these delivery systems. Learning needs to be viewed as a 
dynamic dialogue between the internal cognitions of each student and the sharing of those perceptions with peers: 

Learning is a dynamic process, with communication and interaction the principle 
forces behind this dynamism. A recurring theme (for distance education) is the 
nature of student-to-student interactions and the belief that communication 
among learners is critical to information application. (Willis, 1992) 
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TABLE 1. Description of Matrix Categories and Sub-topics 



1. ) Overview of the ITV Svstem/Network 

- Purpose and Vision: system mission/goals/objectives, intended audience and services 

- System Hardware/Design: the overall technical configuration, transmission capabilities 

- Operations: descriptions of individual sites, their capacities, facilities and equipment 

- Advantages and disadvantages of ITV: its limitations, capabilities and appropriate use 

2. ) E lectro n ic C l a ss room Layout and gqwppiomt 

- Design of the teleclassroom; explanations and diagrams of hardware, seating capacity etc. 

- Instructor's Console: diagram/design of its controls and functions 

- ITV Equipment: overview of basic transmission equipment - operations and functions 

- Auxiliary modes of communication: modems, telephones, e-mail, FAX 

- Instructional aids: Microscopic Camera, Videodisc Player, VCR, Slides, Visual Presenter, Computer(software, 
video signal), Marlite Board, Video CG etc. 

3. ) Su pport System 

- Personnel and Staff: skill positions of an ITV team (technicians, designers, administrators) 

- Roles and Responsibilities of ITV team: specific relationships, duties and processes 

- Initial training: prerequisite skills, familiarity with terminology, equipment 

- Ongoing training: formative development of delivery skills, technical skills, design skills 

4. ) T^hn ka l jj ss oe s 

- Standard Operating Procedures: sequence of basic processes for transmission 

- Troubleshooting: identify equipment difficulties/causes, minor vs. major, "easy fixes" 

- Emergencies/contacts: transmission breakdowns, alternatives, and support 

5. ) Planning and Organizing an Electronic Course 

- Developing a detailed syllabus: course schedule, objectives, criteria, assignments, sources 

- Study guide/ class agenda: lecture outline, learning cues, activities, additional graphics 

- Legal issues/Policies: copyright, release forms, confidentiality 

- Remote Logistics and Support: mail, courier service, libraries, labs, study groups 

6. ) Design and DeveloPinent_oJLj*n Electronic Class 

- Designing ITV graphics: TV colors/aspect ratio, animations, charts, multiple visual aids 

- Questioning strategies: procedural, discussion, evaluative, review 

- Maximizing interactions/feedback: active learning, hands on activities, group work 

- Developing ITV lectures: advanced organizers, more Q/A sessions, discussions 

7. ) Presentation and Delivery Techniques 

- Oral presentation techniques: inflections, pauses, being "off camera" 

- Appearance/Screen Presence: clothing, jewelry, eyeglasses, make-up, camera moves 

- Body Language: movement on camera, use of gestures, eye contact, non-verbal cues 

- Pacing/stimulus variation: slower transitions, more activities and visuals 

8. ) Implementation of the Electranic Course 

- Administration and scheduling: procedures /protocol, registration, constraints, class size 

- Class management: time flexibility, correspondence, on site facilitators 

- Logistics/Distribution of materials: advanced planning-exams, assignments, sources 

- Establishing Interpersonal Rapport: instructor access, student support, visit remote sites 
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9.) Evaluation 

- Formative and Summative methods for the course 

- Formative and Summative methods for the instructor 

- Formative and Summative methods for student performance: alternative assessments 

- Audience Needs/Attitudes: learner analysis, student comfort levels, cultural diversity 

TABLE 2. Results from Individual Matrixes 

Overyiew Matrix of Distance Education Training Materials 

1. Overview of the ITV Svstem/Neiwork 

Purpose/Vision System Hardware/Design Operations/Site facilities Pros/Cons(ITV) 

I 24 I 27 I 27 | 19 I 

2. Electronic Classroom Layout and Equipment 

Classroom Design Instructor's Console ITV Equipment Aux. Communication Instructional Aids 



1 24I 


1 19 | 


24 I 


26 | 19 | 


Sunnort Svstem 


Personnel/Staff 


Roles /Responsibilities 


Initial Training 


Ongoing Training 


1 2 J- 


1 24 


J LL_ 


_J s 1 



4. Technical Issues 



Standard Operating Procedures Troubleshooting Emergencies/Contacts 



1 Ti 1 


16 | 


18 1 


5. Planning and Organizing a Media Assisted Course 




Syllabus/Goals/Criteria Study Guide/Class Agenda 


Legal Issues/Policies 


Remote Logistics/Support 


1 26 1 18 | 


18 


T" 26 1 



6. Design and Development of a Media Assisted Class 



Designing ITV Graphics 


Questioning Strategies 


Maximizing Interactions Developing ITV lectures 


| 22 1 22 j 


I 29 1 29 I 


7. Presentation and Delivery Techniques 

Oral Presentations Appearance/Presence 


ITV Body Language Pacing/Stimulus Variation 


1 21 | 


1 22 | 


1 21 | 22 j 


8. Implementation of the Media Assisted Course 

Administration/Schedules Class Management Distribution/Logistics Interpersonal Rapport 


1 21 1 


1 22 1 


| 24 | 27 | 


9. Evaluation (Formative/Summative) 
Methods for the Course Methods for the Instructor 


Student Performance Attitudes/Needs 


1 19 | 


1 16 | 


1 16 | 14 | 



Amounts are the total frequency of responses per category ( Based on a total of 32 responses) 



TABLE 3. Lesson plan based on the Johnson&Johnson model 



Grade Level: Adults Subject Area: Team Building 

I. Lesson: Conducting an effective meeting 

II. G roup i ng 
Group Size: 5 

Assignment to Groups: existing groups, based on heterogeneous mix of duties 
Room Arrangement: round or oval style table is best 

Materials needed for each Group: copies of the enclosed materials on conducting effective team meetings 

Assigning Roles: Randomly assign all group members to one of the following roles; each member will rotate 
functions for next meeting until all members have participated in each role 

Facilitator : Initiate discussions; Act as gate-keepers: direct the conversation in a productive manner; Keep the 
meeting from digressing by developing and having group adhere to agenda 

Recorder: Write down the meeting proceedings; major decisions, responsibilities, timetables and deadlines for 
projects, notes for next meeting's agenda 

Checker: Seek information and opinions; Clarify or elaborate an idea; Check for understanding of main issues by all 
group members 

Summarizer: Express the group's feelings and ask others to verify that impression; Test for consensus and initiate 
decision-making 

Encourager: Compromise and be creative in resolving differences; Try to ease tension in the group and work through 
difficult matters; Praise and correct others with equal fairness; promote everyone's participation 

III. Set the Lesson: 

Task: Perform team meetings in accordance to a set agenda with timetables and defined roles/responsibilities; have all 
group members read and be familiar with the enclosed text packet "Conducting Effective Meetings". Each member of the 
group will take on one of the described roles for successful team meetings and all members will rotate roles until all 
members have performed each role. All meetings will be conducted under the auspices of an "observer" who will chart 
meeting performance and conduct an evaluation/group discussion of the meetings. Group processing will be conducted 
based on individual responses to questionnaire. Administer final individual test on meeting procedures. 

Positive Interdependence: 

Group Goal : conduct efficient and effective team meetings that meet the timelines, address the agenda issues and 
develop plans of action 

Reward : groups that succesfully rotate roles, pass the Observation checks and all individual members score 90% or 
better on individual test will be given a free lunch/celebration 

Role : each member will complete the role of a Facilitator, Recorder, Checker, Summarizer and Encourager; all are 
responsible for keeping the meeting on time and for checking/understanding final group decisions 
Task Interdependence : all members are responsible for performing their specific roles and promoting the overall 
effectiveness and efficiency of the meeting 

Individual Accountability: Perform all the individual role functions of a group meeting by rotating roles; achieve 
90% on individual test of meeting format, agendas, role/responsibilities 
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Criteria for Succcess: Group successfully performs a series of meetings with all group members rotating and 
performing all the individual roles; encourage equal participation by all members (no dominating); each individual 
member must also achieve 90% correct responses on individual written test of meeting format, agendas, 
role/responsibilities 
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Abstract 

The adoption of alternative pedagogical philosophies in the classroom has led to an increased use of 
technology to expand and enhance authentic , contextual learning environments. Correspondingly these new 
approaches have also led to a growing dissatisfaction with existing evaluation methodologies to evaluate knowledge. 
In this paper the Process/Outcome Evaluation Model (POEM) is proposed to guide in the development of more 
holistic evaluations of both the learning process and the resultant outcomes of that process. POEM consists of four 
components that employ multiple evaluation techniques and strategies resulting in a composite assessment of the 
totality of a learning experience. 

Introduction 

Traditionally, evaluation consists of a measurable test of learner knowledge, usually based on specified or 
implied learning objectives (Carey, 1988). On test day the learner is placed into an artificial testing environment, 
with no access to resources, and asked to answer a series of questions. Test results are calculated, compared to 
acceptable levels of performance, and assigned a grade or ranking. This measure of knowledge essentially evaluates 
only one dimension of knowledge; the ability of the learner to reproduce, recall, or recognize the teacher’s or test 
designer’s knowledge (Choi & Hannafin, 1995). In this scenario, what is learned is determined by what is measured 
(Jonassen, 1996) and since the assessment procedures are restrictive in scope, the learning outcomes and processes are 
commensurately limited. 

In order to reflect the multifaceted nature of learning, many educators believe assessment should encompass 
more than a single dimension of learning (Engel, 1994). This expanded view of evaluation is often referred to as 
authentic, performance-based evaluation (Darling-Hammond, 1994b). Traditional evaluation is described as secretive 
(Wiggins, 1989), expedient (Engel, 1994), and decontextualized (Jonassen, 1991), concentrating on discrete facts and 
rewarding convergent thinking. In contrast, authentic evaluation is seen as public (Wiggins, 1989), cumulative 
(Engel, 1994), and contextualized (Jonassen, 1991), integrating disciplines and enabling the learner to create original 
and unique problem solutions. However, even with authentic evaluation, educators are still confronted with the 
question of how to assess the totality of learning. 

This paper proposes an evaluation model based on the premise that evaluation strategies should reflect the 
full range of the experiences of learning. The model incorporates evaluation strategies that provide a composite 
picture of learning by examining both the learning process and the learning outcome. Termed the Process/Outcome 
Evaluation Model (POEM), the model expands and integrates existing evaluation models by providing tools that can 
help decode, interpret, and assess not only what is learned, but also how the learning occurred. 

Evaluation, Alternative Pedagogies, and Technology 

The continuous and cyclical nature of educational testing makes evaluation pervasive and significant for 
learners and educators. Even though tests serve multiple purposes, including assessing student knowledge, 
teacher/lesson effectiveness, and curriculum content (Carey, 1988), evaluation results most directly effect learners. 
Test scores are used to allow or deny access to educational opportunity. 

Learners are effected by decisions made based on test results that shape their educational future and by a 
curriculum that is dictated by test content. Since the focus is often achievement of specified test results, evaluation 
results can become the driving force behind educational strategies (Worthen & Spandel, 1991). Levels of test 
achievement frequently define what is taught (Haney & Madaus, 1989). 

While supposed or real deficiencies in basic skills of knowledge may be overcome through emphasis on 
traditional subjects such as reading, writing, and computation, critics of the current educational system argue that 
students are unable to apply knowledge and skills learned in school to nonschool situations (see, for example, 
Brandt, 1993). In other words, some educators question whether schools are teaching the skills necessary for the work 
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of scientists, mathematicians, artists, engineers, writers, educators, and the other professionals (Resnick, 1987). 

Constructivist theories of learning recognize the learner as chief architect of knowledge creation through the 
application of unique experiences and beliefs to the learning process. Alternative pedagogies based on these 
assumptions focus on the individual while fostering skills in higher-order thinking and problem-solving strategies. 

Salient features of constructivist learning approaches include an emphasis on a) authenticity (Brown, 
Collins, & Duguid, 1989; Cronin, 1993); b) group work (Savery & Duffy, 1995; Slavin, 1991; English & Hill, 
1994); c) learner definition and control of the learning experience (Jonassen, Campbell, & Davidson, 1994; Kinzie & 
Sullivan, 1989); d) the teacher as guide, facilitator, coach (Cognition &Technology Group at Vanderbilt, 1993); e) 
divergent learning outcomes (Bransford, Sherwood, Hasselbring, Kinzer, & Williams, 1990); and f) the use of 
supportive learning environments that are, in themselves, authentic, ecologically valid, and learner-centered (Krovetz, 
Casterson, McKowen, & Willis, 1993; Newmann & Wehlage, 1993; Stepien & Gallagher, 1993; Morrison & 
Collins, 1995; Wilson, 1995; Laszlo & Castro, 1995). Frequently, the use of ill- structured, unresolved social 
problems such as devising a solution to the nuclear waste problem form the basis of constructivist learning 
environments (Stepien & Gallagher, 1993) where learner solutions are evaluated not by their adherence or 
nonadherence to factual data, but by how well the solution solves the problem being investigated. 

Technology figures prominently in these learning environments since technology enables both knowledge 
exploration (Jonassen, 1988; Locatis, Letoumeau, & Banvard, 1989; Marchionini, 1988) and knowledge creation 
(Harris & Cady, 1988; Jonassen, 1988; Jonassen, 1996; Lehrer, Erickson, & Connell, 1994) without the traditional 
reliance on teacher and textbook as primary providers of knowledge. Technology is adaptable to less traditional 
educational purposes including individual learning, hypermedia development, and information exploration. In 
addition, technology can facilitate evaluation by gathering continuous information of learner progress, providing a 
readily usable storage device for evaluation results, and creating realistic microenvironments where learners are asked 
to solve problems that mirror reality. 

The incorporation of alternative pedagogical models has begun to shift the emphasis away from simple, 
one-shot evaluation measurements to a broader array of assessment methodologies. Alternative forms of evaluation 
tend to encourage the use of multiple vehicles for showcasing student achievement including portfolios, skill 
demonstrations, student performances, artistic works, and computer-related projects. 

Although much attention has been directed to alternative forms of evaluating student progress (Darling- 
Hammond, 1994a; Darling-Hammond, 1994b; Engle, 1994), educational evaluation, in practice, still generally 
focuses on only one dimension of learning; the learning outcome. Evaluating the learning process must also be an 
integral part of evaluation strategies (Choi & Hannafin, 1995; English & Hill, 1994; Jonassen, 1991; Jonassen, 
1996; Kumar, 1994; Kumar, Helgeson, & White, 1994; Marchionini, 1988; Webb, 1995). 

Evaluation of Learning Process and Learning Outcome 

Jonassen (1988) defines learning as the reorganization of cognitive structure, or, the expansion of the 
learner’s semantic network. This definition implies that learning produce divergent, not convergent learning 
outcomes. Further, if divergent learning best occurs when learners are able to incorporate new information into their 
preexisting semantic networks, then allowing learners to determine how best to learn may be preferable to over- 
structuring that experience for the learner (Kinzie & Berdel, 1990). 

One way learners structure their learning experience is by exercising control over various lesson attributes 
including instructional sequence, pacing, and options. Learner control removes the learner from a prescribed, 
regimented, predetermined lesson sequence and allows freedom of movement in and between lesson components. This 
freedom often enables the learner to structure the learning experience in a way that may best facilitate fusing new 
information with preexisting knowledge. 

Learning can be differentiated into learning processes, or how the learning occurs, and learning outcome, or 
what is learned. The strategies and mental processes used for learning differ from what is learned. Learning processes 
are internal, unique to the learner, and difficult to evaluate with traditional testing instruments (Kumar, et al., 1994). 
Learning outcomes, on the other hand, can be externalized by the learner and are normally seen as a product of the 
learning situation. Learning outcomes can be as simple as the learner’s performance on a forced-choice test, or as 
elaborate as a learner-generated hypermedia program. Learning outcomes are generally more easily evaluated than 
learning processes since outcomes are tangible and processes tend to be subtle and often idiosyncratic. 

Separating “hard,” statistically measurable, reliable, and valid evaluation criteria from “soft,” interpretive, 
aesthetic, and affective evaluation criteria provides a classification system to evaluate outcome and process in two 
dimensions. The resulting evaluation matrix contains four categories of measurements: hard-outcome, hard-process, 
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soft-outcome, and soft-process. These four measurements comprise the framework of the Process/Outcome 
Evaluation Model (POEM) (see Figure 1). By analyzing data from each measurement component singly, then 
collectively, evaluation using POEM becomes more comprehensive than traditional outcome-based testing strategies. 
Evaluation takes on a broader perspective, investigating and synthesizing the totality of the learning experience. In 
the sections that follow, each quadrant of POEM will be described and examples of each evaluation type provided. 

Figure 1. The Process/Outcome Evaluation Model 
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Hard-Outcome Evaluation 

Learning outcomes are tangible representations of what the learner has learned. In today’s educational 
climate, a learning outcome can be a multiple-choice test, an essay, a collaborative oral report, a written report, an 
art project, or any number of products or performances that adequately measure mastery of the goals and objectives of 
the learning experience. 

The hard-outcome evaluation component will contain answers to questions that are driven by learning 
objectives and whether or not the learner achieved a satisfactory level of knowledge acquisition or demonstratable 
skill. A wide variety of tests can be employed ranging from forced-choice tests, to criterion checklists containing 
relevant skills for specific activities, to criteria for observational tasks, to problem-specific rubrics. 

Criterion- and norm-referenced tests are typical hard-outcome instruments. However, the tendency to envelop 
testing situations in secrecy where test takers receive little information prior to the test and minimal feedback on test 
results hinders open discussion about test content and test outcomes (Wiggins, 1989), thereby limiting their value as 
evaluative measurements (Berliner, 1992). 





An added dimension to hard-outcome evaluation is the use of multiple evaluators, resulting in juried 
evaluations of products, such as portfolios. Along with improved reliability and validity produced by multiple raters, 
learners can be exposed to multiple sources of feedback and criticism. In this way, evaluation becomes a partnership 
where learners continue with the learning process as cognitive apprentices. 

Hard-Process Evaluation 

Hard-process evaluation consists of gathering and analyzing objective, quantifiable data on how learning 
occurs. Examples include counting the number of times a learner attempts a problem solution; tallying the number 
of interactions between students; tracking how often students follow prompts, guidance, and suggestions; and 
recording the learner’s path through instructional materials. 

Teachers often gather hard process data formally or informally by tabulating learning conditions and learning 
readiness in their classrooms. A teacher may categorize the types of questions being asked in order to induce where 
learners are in the learning process. Constant reference to printed material, quizzical looks, and student conversations 
are other examples of actions educators can count to aid in evaluating the learning process. The use of technology 
can greatly facilitate hard-process evaluation. If, for example, hypermedia is selected as part of the instructional 
medium, audit trail data provides insight into questions concerning the breadth and depth of a learner ‘s path, time 
spent at each learning activity, number of times an activity is accessed, and how often program features such as 
notebooks, help screens, and scrapbooks are accessed. Collection of hard process data not only contributes to 
individual learner profiles, but also assists in analyzing lesson strengths and deficiencies. 

The collection of quantifiable data for path analysis is relatively simple within a hypermedia environment. 
However, path analysis tends to produce large amounts of data per individual learner, and unmanageable datasets for 
groups of learners. If audit trail data is used to analyze the learning process, then methods to organize the massive 
amounts of audit trail data collected must be defined. 

Misanchuk and Schwier (1992) offer four ways to represent quantitative audit trail data including audit trail 
trees, where multiple paths are successively layered on a side-ways tree-like structure, visually reflecting 
commonalities between learner paths, with the most commonly accessed paths appearing as the thickest branches. 

Flynn (1994) used audit trail data to discern “navigational styles” of learners. By recreating and comparing 
paths across learners, four broad categories of navigational styles were identified: a) browsers, b) explorers, c) 
investigators, and d) novices. Each navigational style was defined by how learners accessed program components. For 
example, a learner who consistently moved randomly in the hypermedia lesson was classified as a “novice,” whereas 
learners who went back to screens or activities already accessed and then systematically delved deeper into a specific 
category were classified as “investigators” (Flynn, 1994). 

Based on actual user interaction, audit trail data is valid, measurable, and reliable. The hard-process 
evaluation strategies based on hypermedia-related audit trail path analysis provide meaningful measurements about 
the learner’s journey through hypermedia in a quantifiable way. 

Technology-based observations accomplished through adaptive instructional systems rely on the computer’s 
pre-programmed ability to interpret learner responses to allow for diagnostic assessment and continuous advisement 
(Tennyson, 1984). This approach results in lesson sequences tailored to the individual learner. Lesson options are 
determined by a historical interpretation of what a specific learner has done and is likely to do. Adherence to and 
deviation from prescribed sequences can be tracked and analyzed as part of hard-process evaluation strategies. 

Soft-Outcome Evaluation 

Soft-outcome evaluation encompasses the subjective assessment of learner-generated products and the 
experience in total. These evaluation strategies include criterion checklists, expert appraisal, and self-reflection. In 
general, for soft-outcome evaluation to be valid, products must reflect the outcome of solving authentic tasks. A 
portfolio containing an array of products focusing on the solution to a problem is one example of the type of 
outcome evaluated through soft-outcome strategies. 

Soft-outcome evaluation relies on the expertise and complexity of human evaluative interactions. 
Engagement in ongoing, evaluation-driven dialogue needs to occur between learner and evaluator, between learner and 
groups of raters and critics, between learner and peers, and between learner and experts. These interactions should 
assess learner conceptual understanding through the provision of meaningful and pertinent guidance and advice. 
Learners and their peers, for example, should establish ways to build on each other’s strengths while striving to 
improve areas of weakness. Learners also need to be given time and encouraged to reflect on the learning activity, 
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focusing on what worked and what did not, unresolved questions, and how this learning experience may transfer to 
other problems. 

Historically, the competitive nature of schooling stifles evaluative dialogue. Learners are usually motivated 
to out-perform their peers, not to help them do better. The use of strategies such as cooperative and collaborative 
learning can help foster discussion in the classroom about the importance of understanding and defining the problem 
to the solved, attributes of problem solving strategies, consequences of planning or not planning activities, and the 
efficiency and effectiveness of various problem solutions. These discussions, while evaluative in nature, do not 
emphasize the correctness of a problem solution, but rather stress the strengths and weaknesses of solutions when 
viewed from multiple perspectives. 

Another emphasis of soft-outcome evaluation is assessing learner ability to move onward. Questions to be 
pursued in this regard need to be contextually sensitive, reflective of a particular learning activity. Has the learner 
demonstrated the desire or inclination to broaden the inquiry? Can the learner actively and effectively debate a 
position or belief? What motivates this particular learner? What has the learner done to self-critique his/her 
experience? Combining observation and conversation to derive answers to these questions gives educators a solid 
foundation from which to draw inferences about learning outcomes. 

Soft-Process Evaluation 

Soft-process evaluation strategies employ qualitative measurement techniques. The process of learning is 
internal and often difficult to evaluate without involving the learner in some type of dialogue about what he or she is 
doing, or has done while in a learning situation and why these activities were chosen. For example, correlating 
learner activity with stated intentions during a problem-solving activity can help identify rationales for learning 
decisions and clarify the learning process (Kumar, et al., 1994). 

Kumar (1994) focused on how hypermedia can be used as a “tool for following the learner’s cognitive 
processes” (p. 60), and thus evaluate how learning occurs. Kumar (1994) suggests using the navigational path as the 
basis for understanding the learner’s problem-solving process since it reflects all the choices and decisions made by 
the learner while solving a problem. 

Soft-process evaluation strategies help assess what learners are doing while actively engaged in a learning 
situation. Observational skills are critical in order to better discern and interpret activities such as student 
interactions; the use of ancillary materials, such as reference books and notepads; and the iterations involved in 
building and refining skills, such as when students create multiple versions or drafts of a project or paper en route to 
a finished product. Observing learner behaviors, talking with learners about what is observed, and encouraging 
learners to identify when, where, and why they made specific choices all contribute to evaluation. In this sense, 
learner behavior is used to infer how the learner is learning. 

Additionally, soft-process evaluation strategies are congruent with knowledge engineering, where post- 
instructional changes in a leamefs semantic network are discussed and interpreted (Jonassen, 1996). Comparing a 
learner’s semantic network over time highlights the complexity and interrelatedness of ideas and concepts. 
Computerized tools are available that create a graphical representation of a semantic network, often called a 
“cognitive map” (Fisher, 1990; Jonassen, 1996). A cognitive map represents ideas or concepts as nodes filled with 
text and/or graphics, and relationships or “links” between ideas and concepts are represented by appropriately labeled 
lines (Jonassen, 1996). Cognitive maps are unique to each learner and, since they represent a specific learner’s 
knowledge, they must be created by the learner whose knowledge they represent. This knowledge engineering 
technique may be most beneficial if the learner is able to dynamically alter his/her cognitive map while in a learning 
situation. 

Harmon (1992) theorizes that navigational strategy exhibited by a learner is related to his or her semantic 
network and that the navigational strategy employed by a learner is influenced by preexisting knowledge. Harmon 
and Dinsmore (1994) examined how learners “form semantic association^]” in a hypermedia environment. They 
found that the types of associations, or links, made by learners in hypermedia closely resembled the types of links 
that comprise semantic networking schemes (Harmon & Dinsmore, 1994; Fisher, 1990; Lambiotte, Dansereau, 
Cross, & Reynolds, 1989). 

The interrelationships between navigation, hypermedia, and semantic and cognitive representations may be 
best understood in terms of metacognition. Duell (1986) defines metacognition as knowledge about and regulation of 
the process of knowing. As learners struggle with choosing appropriate problem-solving strategies, planning 
activities, and monitoring progress toward possible solutions, soft-process evaluation can be instrumental in making 
these metacognitive skills more apparent and available to the learner. Suggestions can be given, guidance offered, and 





questions asked that will focus the learner’s attention on what he/she is doing. Soft-process evaluation enhances 
metacognitive awareness by helping learners direct attention to how learning occurs with the ultimate aim of helping 
learners learn about themselves as learners. 

Conclusion 

Viewing POEM as a continuum, reliability and predictive validity increase as evaluation strategies move 
from “soft” to “hard” categories. POEM stresses an equilibrium between objective/quantifiable and 
subjective/qualitative evaluation approaches. The value of the model is its depiction of a holistic framework for 
evaluation. 

Evaluation must move away from its competitive nature and toward a more humane treatment for 
measuring student knowledge and ability. Unfortunately, teachers-in-training get little or no formal instruction on 
comprehensive evaluation techniques and strategies. 

Following the direction of Eisner’s (1983) educational connoisseurship and educational criticism model, 
teachers and learners alike can benefit from embracing a critical connoisseurial outlook and understanding of 
evaluation and how best to measure learning experiences in meaningful ways. Becoming comfortable with these 
techniques takes time and practice, coupled with mutual respect between all involved in the evaluation process. 
Honest and open discussions with multiple evaluators where the learner is encouraged to question opinions and 
critically judge his/her own work will build better learners, better evaluators, and more humane and authentic 
learning environments. 

Evaluation profoundly impacts American education. Throughout the schooling experience, learners are 
subjected to countless tests, quizzes, and examinations that purport to measure what has been learned. According to 
many educators the emphasis on tests has usurped the mission of schools to such an extent that the tests themselves 
control what teachers teach. Of course the problem is not this tendency to “teach to the test,” but the quality and 
unidimensionality of the tests themselves (Wiggins, 1989). 

POEM provides a framework to encourage educators and instructional designers to incorporate a variety of 
evaluation components within constructivist learning strategies. POEM reflects the importance of multiple 
evaluation strategies and proposes ways technology can be used to supplement and complement traditional evaluation 
efforts. 
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Abstract 

Advertisers are taking advantage of the relatively low cost and great exposure of the Internet. The 
proliferation of advertisements on the World Wide Web (WWW) attests to that. Recent attention to gender equity in 
computer environments as well as in print-based and televised advertising for technological products suggests that 
gender bias in the computer environment continues. This paper examines gender messages within WWW 
advertisements. The authors show that advertisers carry the same gender stereotypes from old media into the new 
WWW environment , thus perpetuating the chilly climate for women into yet another territory. 

Computer technology offers great potential for human empowerment. It influences every aspect of our 
daily lives; touching our work, home, and school environments, as well as our entertainment delivery, and 
communication patterns. More and more people have access to an unlimited and constantly updated global 
information resource which carries with it the edge over others who do not have such access. This can be 
particularly influential on educational and employment opportunities. The personal, social, and economic power 
offered by the information age affects the worldwide spectrum of humankind more than any other industrial or 
technological advance in modem history. The effect of computer technology on education alone has been 
remarkable. It provides more than just a new way of teaching; it has become an essential part of educating students 
in life skills necessary for post-school living (Naron & Estes, 1996). 

“Madison Avenue” marketers may consider themselves to be working in the best interest of American 
society by targeting computer technology sales to the upper socio-economic white male, the stereotypical computer 
buyer and user (Lockheed, 1985; Sanders, 1985; Wilder, Mackie, & Cooper, 1985). But unless computer technology 
is distributed across all levels and subgroups in American pluralistic society, it may further enhance the inequities 
that exist among cultural subgroups (Campbell, 1985; Fisher, 1984; Hawking, 1985). Several scholars have 
investigated 

Equity in advertising does not exist in the media, especially in the area of technology. While gender 
stereotypes have evolved over the years, televised programming and advertisements still perpetuate a code that 
identifies feminine and masculine characteristics, with women depicted as feminine and showing their beauty, grace, 
style, and sexual attractiveness, while men are associated with the outdoors, sports, cars, relaxing, or entertaining at 
home (Rutherford, 1994). Further, this stereotypical ideology has spread into the advertising that is associated with 
computer technology, depicting men as high-powered computer users while women are portrayed in subservient roles 
or even as sexual decorations in computer advertising (Knupfer, 1996). Even the clip art imagery used in the large 
majority of popular business and educational software perpetuates this stereotype, with males depicted in leadership 
and authority roles, while women are depicted in subordinate roles (Binns & Branch, 1995). Thus the software itself 
is permeated with subtle gender bias. 

In particular, advertising does not provide for the development of female-computer or minority-computer 
identities despite the fact that significant numbers of females and minorities are now employed in the technological 
field, and many more use microcomputers in their jobs or hobbies. Microcomputers are no longer the private 
domain of the white male, nor should they be. Advertisements in all media should equitably represent the various 
computer users in a manner that is free of language bias and gender stereotypes. Further, because of the potentially 
high level of individual empowerment associated with computer technology, the advertisements should represent the 
racial, cultural, and gender subgroups of society. 

Studies on computer magazines read by adults reveal that these magazines reinforce stereotypes by 
repeatedly providing the message that computers are for white males, that females using computers should be in 
clerical or other subordinate jobs, and females are sex objects to be used as attractions to advertise computer products. 
As these messages are reinforced over and over again, they may become so internalized that they are passed on from 
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adult to adult in general attitudes and employment practices, and from adult to children in communicating individual 
expectations (Marshall & Bannon, 1988). 

This prevailing message of gender bias in visual messages and stereotyped advertising has made its way to 
the newest advertising media, that of electronic advertising on the World Wide Web (WWW). Now the persuasive 
and biased nature of advertising has the ability to reach a larger audience, reinforcing the stereotypical vision of 
computers as male domain to the masses. Further, because of increased use of the WWW in schools, this message 
permeates the boundaries of that protected environment as it goes beyond the home and workplace, into the 
educational system as well. We must then question how much this will affect educational opportunities for females 
as well as their resulting employment potential. 

Students receive daily cultural cues that reinforce the stereotype of computers as a male domain. Such cues 
come from a number of sources including peers, parents, teachers, television, movies, magazines, and literature. 
One source of the cues that has given rise to such stereotypes may be the very images that the computer displays, 
including WWW banner advertising. The images presented may reinforce a particular stereotypical view about gender 
roles related to technology and to life in general. 

Purpose 

The purpose of this study was to extend previously reported research on gender equity in computer 
advertising by examining the advertisements used in the new WWW medium. The study examined the type and 
number of visual images used in WWW banner advertisements, gender roles depicted within them, and whether these 
advertisements contribute to the established stereotypes regarding the roles of computer users. 

Web banner ads are 1” x 7” in size and are interactive graphical links to specific or related advertiser’s web 
sites. They reside on search engine pages that provide links to specific information available on the WWW. 

Earlier studies have shown that a major source of images, computer clip art, routinely under represents 
women and minorities in both numbers and roles (Binns & Branch, 1995; Brownell, 1993; Brownell & 
McConnaughy, 1990; Brownell & McConnaughy, 1991). The majority of clip art perpetuates stereotypes based on 
cultural, racial, and gender stereotypes by neglecting minority populations and by depicting men and women in 
limited roles. Men are typically depicted in leadership and authority roles, while women are depicted in subordinate 
and nurturing roles (Binns & Branch, 1995). Men are shown in work situations or using technology, and women are 
portrayed as either older and overly-grandmotherly, or young and overly-sexy. 

Other studies have shown that in both print and video, which shapes the advertising marketplace, most 
advertising tends to depict women in stereotypical roles. Women, when featured in advertising, tend to be 
represented in select feminine roles such as wife, mother, nurse, teacher, or secretary. Studies of the illustrations in 
popular computer magazines report that men have had the feature role (Sanders, 1985; Ware & Stuck, 1985). 
Women appearing in these same computer magazine illustrations were sometimes depicted as sex objects while men 
were never shown as such. 

Local and national political leaders, educators and parents often see technology as the major answer to 
improvements in education. They frequently claim that by offering greater access to information and computer 
technology that children will do better in school and in life. If used properly, Internet technology as a learning tool 
can be beneficial in education. But at the same time, it opens a door to a world in which gender bias and 

stereotypical attitudes prevail as a norm. Although great strides have been reported concerning equity, studies of 
current practices indicate a serious problem still exists (Binns & Branch, 1995; Couch, 1995; McCormick, 1994). 



Method 

The Internet sampling was done on a Power Macintosh computer using Netscape Navigator, version 2.0. 
Banner ads were selected from topical pages in the five search engine services available with Netscape. Specifically, 
the study employed the Excite, Infoseek, Lycos, Magellan, and Yahoo search engines. Banner ad site selection was 
done by random sampling from the web over several months because of the periodic changing of the advertisements 
and the likelihood that the ads would be repeated in given topic areas of sampling. Table 1 lists the topic and 
subtopic headings common to the five search engines used in the study. 
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Table 1. Topic and Subtopic Headings of Common Search Engines 



• Arts & Humanities 


• Computers 


• Environment 


• News 


• Recreation 


• Business 


Computing 


• Food 


Information 


• Society 


Career 


Software 


• Government 


Reference 


Community 


Employment 


Technology 


Elections 


The World 


Daily Living 


Investing 


• Education 


Politics & Law 


• Personal Pages 


Pop culture 


Money 


Colleges 


• Health & Medicine 


• Regional 


Roads Less Traveled 


Real Estate 


• Entertainment 


• Kids 


• Science 


Social 


Small Business 


Games 


• Mathematics 


• Shopping 


• Spirituality 


• Communications 


Movies 




• Sports 


• Travel 



Random sampling was conducted on sites in each of the five search engines until a minimum of 10 different banner 
ads were found. Not all topic headings in any specific browser were used in this study due to time constraints and the 
size of the population. Ads only with gender specific illustrations or graphics were selected as part of this study, 
thus those ads that did not depict people or depicted people in such as way as to make gender indeterminate were not 
used. Over 700 search page sites were visited while gathering over 50 banner advertisements for review. Electronic 
snapshots were taken of the banner ads and web-site location to preserve the ads intact for this analysis. Qualitative 
analysis of the advertisements was applied, based upon what was found. 

Findings 

Most of the visual representations of women in WWW advertising is a reflection of the current trend used in 
other media. The ads tend to depict women in the supportive or subordinate role. Consistent with other 
commercial advertising formats, males were found in the leadership and authority roles, while females were generally 
portrayed in more passive roles. When women were the only individuals in the advertisement, the ad copy usually 
reflected a biased message toward the abilities and function of the women in their roles as professionals. 

A series of advertisements for used for AT&T WorldNet Services serves as a fundamental example of bias 
depictions in Internet ads. All of these AT&T ads were found under the same major topic heading, Education, in the 
Yahoo search services. 

In Figure 1, the placement of the woman illustrated in the ad is one where she is juxtaposed behind the 
male figure next to her, giving the appearance of a supporting role to her and the leadership role to him. If their 
positioning were reversed, a different message would be illustrated regarding the role of both the man and woman on 
the right side of the page. The woman would be perceived as to being in the leadership role or in equal partnership 
to her male counterpart. It would appear that this ad provides a gesture of acknowledgment of women working with 
technology, but in a deliberate and assigned supporting role. The ad would present a different meaning if it showed 
only the woman in the illustration on the right. If the ad only showed the male illustration, there would be no 
change in the message conveyed. 

Figures 2 and 3 are the only advertisements in which women only were illustrated. But they were not 
depicted in a working or professional manner, as opposed to their male counterparts in Figure 4. The women in 
Figure 2 appear to be dressed in a professional manner, but their body language indicates a more relaxed setting, 
talking on the telephone over a cup of coffee while reading a report. Also, one does not get a complete sense of 
where these two women are located. Are they at their office desk or at home? On the other hand the two males 
illustrated in Figure 4 give a feeling of personal contact in a meeting situation. The attire of the two men is casual 
but not uncommon for some professional settings. 

A key feature of hidden bias is in the ad’s copy. The conversations between the two individuals illustrated 
in each ad conveys different meaning associated with their gender roles. The conversation of the men in Figure 4 is 
focused on a theme of authority, focused on the concepts of money and control. But the conversations between the 
women in Figures 2 and 3 are focused on more nurturing themes of security and responsibility. In Figure 3, the 
message regarding “provider” would be lost or completely misunderstood, if there was not some indication in the ad 
about Internet services. There are no other cues from the women illustrated to indicated their function or role in the 
use of technology. 
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Figure 1. Placement of Male and Female Image 
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Figure 2. Two Women, with Casual Appearance 
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If the gender was changed on these illustrations would the message portrayed in the ads be different? If so, 
gender bias exists. The issue of bias needs to be addressed, but by who? The companies that advertise on the web, 
the ad developers, the copy editors who create the ads, and the web ad account representatives who place these ads in 
their service could play a role in perpetuating or limiting this type of advertising. 

The majority of banner advertisements follow the pattern set by the more traditional media. As is evident 
in previous studies of magazine ads and televised advertisements, when women are portrayed as the central character, 
they are usually depicted by their “beautiful bodies/’ faceless selected parts ( “bodyism”), or as sex objects. This is 
most prevalent in advertisements for beauty products, fashion, and fitness, see Figures 5-10. 

Figures 5. 6. and 7 are typical ads that depict women. These ads are for women and they emphasize the 
importance of beauty, giving women the message that beautiful is better. While there is nothing obviously wrong 
with these three advertisements as separate entities, they combine to promote the stereotype of women as objects of 
beauty and sex by the mere focus of the ads themselves. The important point is the continued emphasis on beauty 
and attractiveness rather than achievement, wisdom, or making good choices as a consumer. 

Figure 3. Two Women, Subordinate and Helpless to Technology Provider 
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Figure 4. Two Men, Dressed Down but Clearly Discussing Business 
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Figure 5. Woman Featured for cosmetic advertisement 
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Figure 6. Woman Feature for Style Advertisement 




Figure 7. Women Featured for Fashion Advertisement 
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The fitness ads also present some of the most interesting uses of gender role illustrations. In Figure 8, the 
woman’s body is used as a graphic element of text. The concept itself is not new, nor does it seem biased in its use 
until we look at the illustration’s interaction with the other elements of text and color. The placement and color of 
the text “The,” draws the viewer’s eye to the illustration of the woman’s body making the shape of the letter “F.” 
But, at the same time it denotes demeaning sexual connotations to the graphic illustration of the woman. The word 
“The” is red, and its last letter ends up in the woman’s crotch. 



Figure 8. Woman's Fitness Ad with Body as a Graphic Element. 
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Figure 9 shows part of a man’s torso, left shoulder, and arm. The emphasis is on strength. Figure 10 does 
a good job in providing a balance of both male and female illustrations advertising the concept of fitness, but the 
way in which the bodies are posed conveys two different messages. The male image portrays strength and the female 
image suggests sex appeal. Are all men to be strong and all women to be sexy if they are physically fit? Does 
strength translate into male sexual appeal? Although the importance of facial characteristics and general body 
structure is absent from the male side, the fitness ads do suggest that big muscles are important for men, thus 
continuing the physical stereotype for males as well. The difference between the male and female ads is that the 
large majority of male ads do not focus on the physical features. 

One gender-specific “bodyism” found in Internet advertising and web page graphics has been the use of 
eyes. Use of the eyes as an illustrated or graphic element in advertisements seems to be feminine in composition, 
see Figures 1 1 and 12. Color plays a major role and how it is used can be appropriate or abusive. In Figure 1 1, the 
photo illustration of the eyes works well as a supportive graphic element, but use of both a different color and font 
for the word “sex” changes the intended message of the ad from a “helpful resource” to “sex.” 

Figure 9. Man 's Fitness Ad with Musculature “Bodyism " 









Figure 10. Fitness Ad Featuring Sexiness for Women and Strength for Men 
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Figure 11. Balanced Use of Woman’s Eyes, but Color and Font Emphasize "Sex 
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Figure 12. Woman's Eye with Sex Appeal 
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Recording the location of the sampled banner ads was important in observing a relationship of gender-biased 
placement of ads within the Internet. Ads are located under selected topic areas and target specific audiences. Gender- 
specific illustrations, especially in the areas of technology and business do not work well in trying to target other 
audiences. Figure 13 targets a male audience for financial investments and it was located with the content path of 
shopping, then smimwear. Would the advertisers suspect that males might be more inclined then females to look at 
photos of people modeling swmsuits? 

In other situations, repeating of particular ads in specific locations can infer additional hidden or gender-bias 
messages to the visitors of these topical search sites. Figures 14, 15, and 16 suggest that the male audience is the 
target. If so, then clearly a message of male freedom, control, or irresponsibility as well as female passivity and lack 
of control could be the message. 

Conclusion 

The WWW offers a number of exciting opportunities for advertisers to complement other forms of media, 
not replace them (Teague, 1995). The number of print ads in general and trade magazines, and newspapers that now 
include the statement “for more information call or visit our web site” is growing each day. Advertising on the 
Web can be comparatively inexpensive compared to the potential number of customers who would have access to 
your information, and will spawn a dramatic increase in its use as a marketing tool. 

Figure 13. Male Audience Targeted with Ad Found within Shopping , Swimwear Section 



i 



77 



177 




Kaufmann 



IcHckhereJJ 



"One of the Best 
Funds of the 90s 

-Kipfing&r's Mutual Fundi *96 




Sho pping > Clothes & Personal Stuff > Svimvear Revievs 



O 

ERIC 



178 




Figure 14. Woman* s Appeal to Keep in Touch , within Parenting and Babies Section 
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Figure 15. Woman’s Appeal to Keep in Touch, within Men and Fathering Section 
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Figure 16. Woman* s Appeal to Keep in Touch , within Men and Circumcision Section 
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This study raises questions for further research regarding the use of advertising in the WWW. The first, is 
current advertisement targeted toward the characteristics of the majority of the users of the technology associated with 
the WWW? Is this audience any different than that of magazine readers or television viewers? How is the 
advertising the same or different? Is the representation of gender roles portrayed in the advertising media reflective of 
the current skills, level of computer use, and interests, of the majority of women in general or the majority of 
women who currently use the WWW? What is the fallout of the current gender bias practice in advertising on the 
WWW; how does it impact the development of gender stereotyping in the field of computer technology today and in 



the future? How can the growing technological marketplace of the WWW be used to bring about change in 
stereotypical views? How can teachers counteract some of these message stereotypes that students are sure to 
encounter while using the WWW in school? How does one start to educate the teachers to recognize gender bias in 
the ads? Can that be done or are people so used to living with it that it becomes invisible, even in this new format? 
In addition to education, what is needed to break the continuing gender stereotypes that our culture perpetuates 
through advertisements? 

Educators can play a special role in the process of promoting equity in advertising. They provide the 
quality opportunities for students to learn regardless of race, culture, or gender. Educators must expose the practices 
in society that are biased, especially those related to computers as computers and access to information continue to 
gain importance. They can provide research on the problem, attempt to bring about change in the classroom and 
publicly. They can help to educate parents, who have such great influence on their child’s view of gender roles. 
Education can be a potent force in combating the issue of gender bias in advertising and bringing pressure to bear on 
business to be attentive to these problems. But ultimately it will be the moneymakers, the people who make and 
distribute the advertisements, who will need to pay better attention to issues related to blatant and subtle gender 
equity and potential opportunity. 
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Abstract 

Interest in creating productive and meaningful learning environments has been an enduring theme in the 
educational arena. Open-ended learning environments (OELEs) provide contexts to support exploration , 
experimentation , and problem-solving. The purpose of this paper is to create a context for understanding the issues 
and challenges associated with the design and development of OELEs. 

Interest in creating productive and stimulating learning environments is an enduring theme in the 
educational arena. Open-ended learning environments (OELEs), learning environments designed with the intent of 
supporting the development of understanding, are drawing attention in the educational arena. In OELEs, experience 
and context are critical for cultivating cognitive processes that support understanding. Several processes are essential 
in these environments: exploration and experimentation; problem-solving and critical thinking; and angling, or 
viewing from multiple perspectives (Duchastel, 1990). While the creation of learning products or outcomes remain 
important to knowing, processes and strategies are the focal points of growth in understanding in OELEs (Hannafin, 
Hall, Land, & Hill, 1994; Papert, 1993). 

A variety of frameworks can be used to establish OELEs, ranging from the classroom to computer-based 
environments. In classroom settings, OELEs focus on the learner. The instructor adopts the role of facilitator as 
opposed to one of leader. Additionally, the instructor is but one resource from which the learner seeks information; 
peers, parents, those working in the community, as well as print and electronic resources are all important in 
assisting the learner in developing understanding. Technology-based learning environments developed following 
learner-centered principles like their classroom counterparts. However, unlike classroom environments where the 
learner works with the instructor in finding resources to gather information, technology-based OELEs provide a 
built-in range of tools and resources for learning. These resources are used by learners as they build and revise 
understanding. 

The emphasis on cultivating processes and strategies increases the requirements on the learner to 
productively use and learn from open-ended environments. While the instructor or system can assist by guiding and 
facilitating, the learner determines how to use the available tools and resources to access and manipulate information 
deemed relevant (Hannafin, 1992; Spiro, Feltovich, Jacobson, & Coulson, 1991). The individual is critical for 
determining what s/he wants and/or needs to know, for accessing the information in the system, and for deciding 
whether or not the system contains what is needed (Perkins, 1993; Roth & Roychoudhury, 1993). 

The challenges associated with creating open-ended learning environments are considerable. OELEs are a 
milieu of duality. They can be simplistic and focused or complicated, appearing virtually limitless in scope. OELEs 
are empowering, yet exacting; liberating, yet hindering; expansive, yet disorienting. 

Learning and building understanding in an OELE may seem a daunting task, for both the learner and 
instructor. Yet, a greater challenge exists: the design and development of these environments. There are several 
critical, but as yet undeveloped, design and development requirements in creating OELEs. Truly learner-centered 
systems require more than a shifting of control to the learner; OELEs require that the environment be conceptualized 
in ways that make sense to the individual. Designing learning environments which empower learners to use 
available tools and resources in building and evolving self-directed understanding is a formidable challenge, 
particularly when one considers the “compliant” thinking shaped by conventional classroom activities (McCaslin & 
Good, 1992). 

The purpose of this paper is to create a context for understanding the issues and challenges associated with 
the design and development of OELEs. We first discuss the characteristics of OELEs, contrasting them with 
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traditional learning environments. Next, assumptions related to OELEs are presented, along with the associated 
design issues. Finally, a conceptual framework for the creation OELEs is presented. 



Open-Ended and Traditional Learning Environments: Key Differences 

OELEs differ from traditional learning environments in several ways. These differences involve the role of 
context, the importance of multiple resources and tools, and the responsibilities of the learner. 

One of the key differences between the environments is that traditional learning environments do much of 
the work for the learner. While the learner certainly engages in some activities within the system, the activities, as 
well as the context in which those activities occur, has already been selected. OELEs, in contract, directly involve 
the learner in the effort. The learner is actively involved in establishing both the context, as well as the activities 
that will occur. While a framework is established for the learner in an OELE, this is merely a foundation on which 
they build. The scaffolding necessary in building understanding is provided by both the learner and instructor. 

OELEs also differ from traditional learning environments in the importance of resources and tools. In 
traditional environments, these resources and tools are often pre-selected, awaiting use within the pre-defined context. 
Resources and tools hold a different position in OELEs. In these environments, not only is the learner responsible 
for making decisions related to which resources and tools will best meet their needs, they are also encouraged to 
examine multiple resources as they work to angle and integrate the information in building understanding (Duchastel, 
1990). 

Finally, traditional learning environments differ from OELEs in the responsibility level of the learner. In 
traditional learning environments, the learner has minimal responsibility; they attend class, complete activities 
and/or exercises in a variety of media; and are evaluated on their performance. The primary role of the instructor is of 
“sage” while that of the learner is “disciple:” one leads, the other follows. In OELEs, this role-play is very different, 
and often reversed. The learner directs the environment, while the instructor is there to support and guide when 
needed. The learner, in taking on a directive role, also takes on responsibility for what occurs in the environment, 
including the development of understanding. 

These fundamental differences create a unique opportunity to change educational environments to highly 
interactive, engaging situations, for both the learner and instructor. However, the challenges associated with creating 
these environments is not insignificant. In the next section, we present issues for design of OELEs, along with 
their inherent assumptions. 

Assumptions and Issues for Design of OELEs 

Assumption One: Understanding is best achieved when situated in relevant contexts that 

support the learner in connecting personal beliefs and experiences with formal concepts 

One assumption of open-ended learning is that personal beliefs, experiences, and conceptual schemata 
support current, as well as provide the foundation for new, understanding (Hannafin, 1992). Background knowledge 
and experience form the conceptual referent within which new encounters are organized and assimilated (Piaget, 
1976). Background context influences the choices learners make in the environment, the extent to which they 
persevere, and the types of goals they set. Accordingly, learner use of prior experiences as a referent for 
understanding is foundational to the design and development of OELEs. 

Problem-based contexts are established in OELEs to influence how learners make decisions and access prior, 
related experiences. Problem-based contexts often provide orienting scenarios to guide learners in exploring the 
complexities of a topic. Such scenarios often focus on everyday problems (e.g., environmental pollution or 
contamination of drinking water) that can be readily identified by learners (Tobin & Dawson, 1992). In this way, the 
environment establishes a context for identifying unmet needs, accessing prior experiences, and generating plausible 
strategies and solutions. 

Design Issues OELEs require learners to link everyday experiences with formal experiences (Hawkins & 
Pea, 1987). Without problem contexts designed to help learners access related prior experiences (e.g., provide 
science problems involving roller coasters that help learners use prior experiences with roller coasters) or external 
structures to prompt the connection of these experiences, learners will likely fail to bridge the gap between formal 
and experiential. 

At times, however, learner prior experiences are not directly accessible, or often conflict, with formal 
concepts under study. Research in science misconceptions has indicated that individuals develop informal theories 
about scientific phenomena, which may help or hinder their subsequent learning (Driver & Scanlon, 1988). This is 
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common in science and mathematics learning, for instance, where concepts are often abstract or when they are not 
easily represented in the learner’s everyday experience. Newtonian motion, for instance, is based upon assumptions 
that take place in a gravity-free world. Since most learners do not experience gravity- free environments, it is difficult 
for them to connect prior experience to the formal. This provides a limited basis for understanding that, as a 
consequence, is often superficial, inert, and riddled with misconceptions (Carey, 1986). Assisting the learner in 
bridging the gap between everyday experiences and formal learning experiences is critical in the design of OELEs. 

Research examining the development of cognitive strategies while using OELEs also indicates mis- 
application of everyday experience (Hill & Hannafin, in press; Hill, 1997). In these studies, learners were engaged in 
the task of retrieving information from the World Wide Web (WWW), one example of a technology-based OELE. In 
several instances, learners relied on their prior experience in retrieving information in a traditional electronic library 
catalog to inform their decisions. The use of this model was not sufficient to guide the learner as they searched for 
information on the WWW. The model generated was “...messy, sloppy [and] incomplete” (Norman, 1983). 
Assisting the learner in the creation of a mental model appropriate for the environment is a critical design component 
in OELEs. 

In some situations learners may use misconceived prior experiences as the basis for interpreting the 
concepts under study in the OELE. OELEs such as microworlds are designed to allow learners to vary parameters, 
model or represent their understanding, and use simulations to test their validity (Rieber, 1992). In some instances, 
however, the system may be incapable of responding to learner misconceptions that are based in prior experiences if 
they cannot be tested operationally (Land & Hannafin, in press). Consequently, enduring conceptions result that are 
often difficult to alter. Assisting the learner as they develop skills associated with angling, that is viewing from 
multiple perspectives, is another critical design issue associated with OELEs (Duchastel, 1990). 

Assumption Two: Provision of resources and tools help learners connect and manipulate 

understanding 

OELEs increase learner access to both sources and perspectives related to the content under study. Tools for 
constructing and manipulating understanding are used to promote learning that is more concrete and capable of being 
tested. Often, a range of resources are provided that serve as repositories of information (e.g., CD-ROMs, 
encyclopedias). However, environments can also be designed to facilitate the construction of resources by learners. 
For instance, students can learn about fractions by designing and constructing educational software for teaching 
younger children about fractions (Harel & Papert, 1991). Similarly, tools such as Intermedia utilize a networked 
multimedia system where learners construct a "web" of concepts, and share them communally with other students. 
The use of resources, or opportunities to construct resources, provides a rich environment for extending 
understanding. 

Tools, such as spreadsheets or word processors, provide opportunities for user-centered activity. In learning 
environments, tools help learners to manipulate features and processes. Some tools, such as those found in 
simulations and micro worlds, allow learners to manipulate concepts by varying parameters and/or physical models 
(e.g., vary force and direction of an object in space, [Rieber, 1992]). Computerized tools can be used to select text 
for electronic notebooks, create hyperlinks between sources of information, or perform calculations (Hannafin, 
1992). Tools allow learners to test complex theoretical concepts in concrete ways (Hannafin et al., 1994). 

Design Issues Use of tools and resources in OELEs is a learner-directed activity. In this context, tools and 
resources provide an opportunity for use, but do not inherently enhance cognitive activity or skills. Perkins (1993) 
refers to the "fingertip effect” as a misconceived belief that automatic provision of tools or resources spontaneously 
prompts their use in the ways in which they were intended. Previous research has indicated, for instance, that 
learners often do not use the tools and resources provided by the system in ways that promote understanding (Hill & 
Hannafin, in press; Land & Hannafin, in press).. The way in which tools and resources are integrated into the 
environment influences the manner in which they will be used. The “seamless” integration of tools and resources 
needs to be a goal in the design of OELEs. 

Interface design and feedback mechanisms offer useful structures for tool and resources use in technology- 
based OELEs. Interface techniques such as the use of metaphors (see, for example, Tobin & Dawson, 1992), 
prompts for where and how to find information, and required sequences for using and interpreting feedback (Lewis, 
Stem, & Linn, 1993) have been used as methods to facilitate intentional use of resources and tools. Regardless of 
the apparent power and affordances of tools and resources, it is unlikely learners will use them "mindfully" without 
thoughtful facilitation and well-designed interfaces (cf., Salomon, 1986). 



In addition to encouraging the use of multiple tools and resources, OELEs should also be flexible in the 
ways in which the information retrieved and/or generated from these resources and tools is presented. Multiple views 
of the same information for learning complex material is one of the main tenets of cognitive flexibility theory (see, 
for example, Spiro et al., 1991). In order to support the learner in the creating individual understanding, OELEs need 
to allow the learner to manipulate and view the information in ways that best match their needs. Designing this 
flexibility into the environment will help support the learner in linking information in meaningful ways (Borsook & 
Higginbotham- Wheat, 1992). 

Assumption Three: Learners must take more responsibility for monitoring, and reflecting 

upon, the learning process 

To be effective during open-ended learning, learners must also monitor their thoughts and actions. Learners 
interact based upon metacognitive awareness of their understanding and the perceived need to validate or challenge 
their understanding (Perkins, 1993). This includes decisions to pursue additional practice, search for definitions or 
information, test a hypothesis, create a “what if’ scenario, or take notes. Learners must be able to locate, select, 
organize, integrate, and use relevant information if they are to generate products and/or understanding. Similarly, 
learners must evaluate the adequacy of their approaches during open-ended learning (Belmont, 1989). This is 
especially important given the numerous learner control studies which suggest learners often fail to both invoke self- 
regulation strategies and to initiate and direct their own efforts (Steinberg, 1989; Zimmerman, 1989). 

Design Issues Available tools and resources may fail to promote understanding if the OELE does not 
facilitate the needed cognitive or conceptual processes. Learners searching for information on the WWW 
demonstrated how access to a virtually limitless amount of information and resources is not sufficient in the pursuit 
of understanding (Hill, 1997; Hill & Hannafin, in press). Despite ready access at their “fingertips” (Perkins, 1993), 
the majority of the learners were unable to manipulate and use system features to further their own understanding. 

Much of the lack of manipulation was attributed to “doing and not thinking.” What the studies revealed 
was a lot of physical engagement (i.e., pointing and clicking) with the system by the learners rather than reflection 
before action. While “doing,” or enactive mentality, is important, this alone is not sufficient for building 
understanding(Kay, 1990). Iconic mentality (i.e., recognition, comparison, and configuration) as well as symbolic 
mentality which enables sign interpretation and connections to familiar environments in making sense of the 
information, are also critical for developing strategies and processes in OELEs. In these Web-based studies (Hill, 
1997; Hill & Hannafin, in press), the learners’ lack of “sense-making” contributed to under-developed levels of 
understanding (Dervin & Nilan, 1986). This lack of understanding spanned both the information that was retrieved 
as well as the system itself. Guidance and assistance in developing monitoring and reflection techniques must be an 
inherent part of the design of OELEs if the learner is to develop the strategies and processes necessary for building 
understanding. 

Recent design efforts have emphasized activities that induce and facilitate high-level cognitive processes, 
such as hypothesis generation, scientific reasoning, or metacognitive analysis. For instance, the CSILE 
environment is designed to facilitate metacognitive thinking through the use of prompts to generate questions, 
hypotheses, or theories (Scardamalia et al., 1989). Other environments facilitate scientific inquiry or critical 
thinking skills by embedding activities that induce learner hypotheses and observations prior to manipulating and 
collecting data (Lewis et al., 1993). Pedagogical approaches, as well as technological interventions, must embed 
structures for requiring the use of higher-order thinking, reflection, and evaluation. 

Conceptual Framework for the Design of OELEs 

In an effort to begin offering solutions to the challenges associated with designing OELEs, a conceptual 
framework has been developed (see Figure 1). The framework has represents a starting point for integrating 
problems and issues with approaches to designing OELEs; evaluation studies and/or empirical investigation have not 
yet been conducted. 




184 




Figure 1. OELEs: Framework for Design 

The three connecting circles represent three major components associated with OELEs: context, resources, and 

learners. Each of these is closely associated with the three assumptions presented earlier in the text. A fourth area, 
evaluation, has been added to the framework. As in all learning environments, evaluation is a critical component to 
ensure that goals are being reached. In OELEs, evaluation is not only a self-reflection technique for learners; it is 
critical on the part of the instructor to ensure that the needs of the learners are being met. 

In designing OELEs, creating a context conducive to supporting individual needs is critical for supporting 
the development of understanding. This context needs to problem-centered, yet activity-based. The activities should 
also assist the learner in the development of an appropriate mental model, as well as encourage the use of multiple 
perspectives in developing understanding. By taking an additional step, grounding the environment in a problem- 
centered context defined by the learner, the task of bridging everyday experiences to formal learning situations will be 
even easier. 

The provision of multiple resources and tools is another critical component in the design of OELEs. Using 
techniques of seamless integration, as well as following guidelines for well-designed interfaces, will enhance the use 
of these resources and tools. Flexibility should also be an integral part of the design of OELEs. Encouraging the 
learner to use the tools in a variety of ways, as well as advocating individual selection of resources which best match 
their needs, should also be a part of the design and development of these environments. 

The learner is the focus of OELEs. As such, their role in OELEs is a critical component in the design of 
these environments. Developing strategies and processes which facilitate growth in understanding is a primary goal 
of OELEs. To enable learners to reach this goal, monitoring and reflection strategies need to be facilitated and 
encouraged. By facilitating the processes associated with evaluation and contemplation, OELEs can assist the learner 
in the use of higher-order thinking skills. This, in turn, will enable growth in understanding. 

Conclusions and Implications 

The design of OELEs is challenging; in fact, it can be an exceedingly complex undertaking. New 
responsibilities are created for both the instructor and learner engaged in these environments. The instructor can no 
longer play the role of “sage on the stage,” but rather must move into the realm of “guide on the side.” 
Concurrently, the learner must be prepared to take responsibility for developing understanding and learning. 
“Compliant thinking” is the antithesis of what occurs in OELEs (McCaslin & Good, 1992). OELEs are grounded in 






a familiar context; replete with resources, both human and non-human; and rich with activities and interactions 
amongst the participants. “Energetic thinking” is not only essential, but required if OELEs are to reach their true 
potential in assisting the learner in evolved understanding. 

Resolving issues inherent to the design of overall context, resources and tools, and facilitation and feedback 
appear fundamental to advancing our understanding of how to design and develop OELEs. With a more inclusive 
understanding of how to operationalize underlying theoretical assumptions of OELEs, new directions for teaching and 
learning become not only inviting, but feasible. 
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Abstract 

National standards published in 1995 addressed the importance of foreign language education. Five goal 
areas were identified: communication, cultures, connections, comparisons, and communities. This study 
incorporated these five goals and examined the potential of electronic mail in foreign language learning. In addition, 
computer mediated communication and cultural aspects were investigated. 

The subjects of the study were students in the fifth- semester Spanish Conversation and Composition class 
at a large public university in the United States. Two classes, with a total of 35 students, were involved in the 
project. The students of one class (the E-Mail group) used E-Mail to communicate with native Spanish speakers in 
Mexico, while the students of the other class (the non-E-Mail group) did not. The E-Mail communication lasted for 
10 weeks. The purpose of the study was to examine (1) whether the language performance and confidence of these 
two groups differed, (2) what the E-Mail messages involved, (3) what the students ’ attitudes toward the use of E-Mail 
were, and (4) what problems the students encountered during the E-Mail communication. 

The data collection included a cloze test, class writing reports, oral examinations, attitude surveys, 
questionnaires, and interviews. Both quantitative and qualitative research methods were used to analyze the data: 
repeated measure, selective coding, survey design, and interview. The analysis of the quantitative data revealed that 
the students' performance and confidence did not differ significantly between the two groups. For the qualitative 
analysis, the content of the E-Mail messages were categorized into eight topics: culture, language, course 
management, social activity, books, class assignments, desire to communicate, and personal sharing. The findings 
of the analysis indicated that E-Mail motivated students, helped in learning culture, enhanced social presence, and 
assisted foreign language learning. The participants were in favor of the use of E-Mail. 

A follow-up study was conducted one year after the study was completed. One fourth of the participants of 
the E-Mail group continued the E-Mail communication with their pen-pals after the study was over. One student 
was still writing to her Mexican pen-pals when the follow-up study was conducted The findings indicated that the 
study had a positive impact on the students over time. 

E-Mail could be a very useful tool for foreign language learning. It is recommended that educators integrate 
E-Mail into instruction. Since few studies of this kind have been conducted, more research is warranted to learn 
about the best use of this promising technology. 

Introduction 

The importance of foreign language education was addressed in national standards in 1995 (American 
Council on the Teaching of Foreign Languages, 1995). Computer medicated communication (CMC) technology is 
considered to be a good tool for foreign language learning because it enables foreign language learners to 
communicate directly with native speakers and to expose themselves to an authentic culture (Cohen & Riel, 1989; 
Cononelos & Oliva, 1993; Sayers, 1994; Knight, 1994). CMC is based on text, disregarding the lack of social 
cues, such as facial expressions, gestures, and smiles. Research studies on CMC indicated two opposite findings. 
Researchers conducting experimental studies suggested that CMC was cold, unfriendly, and impersonal because of 
the lack of social cues (Rice, 1984; Trevino, Lengel, & Daft, 1987; Hackman & Walker, 1990; Hiltz, Johnson, & 
Agle, 1978; Rice & Case, 1983; Steinfield, 1986). Therefore, CMC was not appropriate for communication and 
personal interaction. In contradiction, researchers conducting field studies claimed that users developed on-line 
community and that CMC was warm, friendly, and personal although CMC lacked social cues (Kerr & Hiltz, 1982; 
Hiltz & Turoff, 1978; Steinfield, 1986; Johansen, DeGrasses, & Wilson, 1978; Foulger, 1990; Gunawardena, 
1995). So, CMC was suitable for personal interaction and communication. 

According to Short, Williams, and Christie (1976) and Walther (1992), the fewer codes (social cues) that are 
available in a medium, the less attention is paid by the user to the presence of other social participants. In other 
words, users participating in CMC would pay little attention to their partners. Consequently, CMC messages 
contain little personal information, and messages are task-oriented and impersonal. The purpose of this study is to 





investigate the potential of E-Mail in foreign language learning and to examine the relationship between CMC and 
social presence. 

Research Questions 

The following five research questions were asked : 

1. Does the reading, writing, and speaking performance of the students using E-Mail in class differ from the 
performance of those students not using E-Mail in class? 

2. Does the confidence of the students using E-Mail in class differ from the confidence of those students not using 

E-Mail in class? 

3. What does the content of the students’ E-Mail messages involve? 

4. What are the opinions of the students and instructor about using E-Mail in foreign language instruction? 

5. What problems do students encounter during communication with their pen-pals via E-Mail? 

Student Samples and their Mexican Pen-Pals 

The sample for this study consisted of the students in two Spanish 313 (Spanish Conversation and 
Composition) classes. The students taking that course were selected because their language skills were sufficient for 
E-Mail communication with native speakers. In addition, they were learning language skills and culture. They were 
taught on the same days by the same instructor, who was a Spanish native speaker. Both classes had around 20 
students. One class learned and used E-Mail during the semester, while the other class did not. 

The Mexican pen-pals were 36 university students of a mathematics class in Instituto Tecnologico y ds 
Estudios Superiores de Monterrey. Via personal contact, the professor of the mathematics class expressed his 
interest in the project and was willing to have his students to communicate with the U.S. students in Spanish. 

Procedures 

First, the researcher explained her study to the students of the two Spanish 313 classes and invited them to 
participate in the study. All of the students were willing to participate in the project. They filled in an attitude 
survey form (see Appendix A) and took a cloze test in class (see Appendix B). The students of the E-Mail group 
received step-by-step instructions on using E-Mail, but due to time constraints they were only taught the commands 
that were necessary for the study. The non-E-Mail group did not receive E-Mail training. 

To encourage the students to write more, the instructor agreed to give extra points up to 10% of the total 
scores for participating. For the students of the E-Mail group, the extra points were based on the number of the E- 
Mail messages sent to the Mexican pen-pals and the amount written in the E-Mail messages. To gain extra points, 
the students of the E-Mail group had to send copies of their E-Mail messages to the researcher. Similarly, the 
students of the non-E-Mail group also received extra points if they wrote a summary of an article which they read 
outside of class time. Their extra points were based on the number of articles they read and the amount of 
summaries they wrote. To receive extra points, the students had to submit their summaries to their instructor. The 
instructor and the researcher together decided on number of extra points for each student. 

During the beginning of the E-Mail communication, the U.S. students’ letters were returned from Mexico 
as undeliverable mail. To continue the project, the researcher mailed the U.S. subjects’ E-Mail addresses to the 
Mexican professor, and the Mexican students initiated writing to the U.S. subjects. The U.S. subjects 
communicated with the pen-pals by using the "reply” command, and the communication went smoothly. The E- 
Mail communication lasted for 10 weeks. 

Toward the end of the semester, the students in both the E-Mail and non-E-Mail classes took the same cloze 
test and survey as at the beginning of the project. The E-Mail group filled in an additional questionnaire (see 
Appendix C) to share their opinions about using E-Mail in their learning. Afterwards, a telephone interview was 
conducted to clarify the responses on the questionnaire. Finally, the instructor was interviewed to share his opinions 
about the use of E-Mail in his class. 

Data Analysis 

Both quantitative and qualitative research methods were used to analyze the data. The data analysis consisted 
of three major parts. The first part included the use of a t-test to compare the language performance (reading, 
writing, and speaking) and confidence of the E-Mail group with the same characteristics of the non-E-Mail group 
(research questions one and two). The data were based on scores of the cloze test, papers written in class, oral 
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examinations, and the attitude survey. The second part involved coding and analyzing content topics in the E-Mail 
messages (research question three). The data were based on the E-Mail messages which were sent to the Mexican 
pen-pals by the U.S. students. The third part covered reviewing and analyzing the data collected from the E-Mail 
messages, survey questionnaires, and interviews (research questions four and five). 

Results 

The five research questions were about the students' language performance, confidence, E-Mail content, 
opinions of using E-Mail in learning, and problems encountered during the E-Mail communication. 

Language Performance and Confidence 

The p value in the following table indicated that no significant difference was found between the E-Mail group and 
the non-E-Mail group. 



Table 1. Results of Language Performance and Confidence between the two Groups 



Topics 


Reading 


Writing 


Speaking 


Confidence 


p value 


.50 


.56 


.76 


.30 



> 0.05 

E-Mail Content 

Table 2. The Number of E-Mail Messages and their Total Number of Words 



Student 


Number of E-Mail Messages 


Total Number of Words 


i 


12 


3384 


2 


13 


1904 


3 


21 


4230 


4 


7 


846 


5 


4 


635 


6 


0 


0 


7 


7 


846 


8 


16 


2538 


9 


20 


4865 


10 


6 


1269 


11 


17 


2115 


12 


6 


635 


13 


7 


1058 


14 


10 


1269 


15 


12 


2115 


16 


8 


1481 


17 


5 


635 


18 


0 


0 



The table above listed the number of each U.S. student’s E-Mail messages sent to his or her Mexican pen- 
pals and the total number of words written in the messages. Two of the 18 students did not write E-Mail at all. 
Two of them sent 20 or more messages to their Mexican pen-pals and wrote more than 4,200 words during the 10- 
week E-Mail communication. 

The content topics of the E-Mail messages included culture, language itself, desire to communicate, 
personal sharing, course management, social activity, books, and class assignment. "Culture" referred to all aspects 




191 



190 



of the life of the people. It included people’s customs, manners, values, and beliefs. "Language accuracy" referred to 
the E-Mail messages about language, for example, grammar, language use, semantics, or syntax. ’’Desire to 
communicate” referred to the E-Mail messages which expressed the desire of the U.S. students to communicate with 
their Mexican pen-pals. "Personal sharing” referred to the E-Mail messages which contained discussion about 
personal thoughts that people often prefer to share only with their friends. ’’Course management” was related to 
discussion about extra points, due date, scores, and examinations. "Social activity” was about invitations or plans 
for activities. For example, students used E-Mail to invite people for a party or organize a trip. “Book” listed some 
students’ thoughts and feelings towards certain books and authors. "Class assignment” referred to the E-Mail 
messages that the students used to collect information which they needed for their assignment. The messages of the 
first four topics (culture, language accuracy, desire to communicate, and personal sharing) frequendy appeared in the 
E-Mail messages. 

Opinions on E-Mail Use 

The following table summarized the attitude of the students of the E-Mail group toward using E-Mail in 
foreign language learning. 

Table 3. Attitude of Students Toward Using E-Mail 



Opinions 



Mean 



E-Mail has increased my motivation to continue learning Spanish. 3.44 

I learned a lot from my pen pal(s). 3.50 

E-Mail has increased my confidence about my Spanish ability. 3.63 

E-Mail has enhanced my understanding of Spanish-speaking culture. 3.63 

E-Mail has improved my ability to write in Spanish. 3.69 

E-Mail has improved my ability to communicate in Spanish. 3.75 

E-Mail has improved my ability to read in Spanish. 3.86 

I like to communicate with Spanish native speakers via E-Mail. 4.3 1 

E-Mail should be integrated into Spanish language instruction. 4.38 

E-Mail was a positive addition to Spanish 313. 4.56 



Note. The scale is from one to five with five corresponding to a student’s strongest agreement with the statement. 

The students rated high the statements "E-Mail was a positive addition to Spanish 313” and "E-Mail should 
be integrated into Spanish language instruction." They also remarked that they enjoyed the communication with 
Spanish native speakers very much. They rated as fairly high statements about E-Mail improving their language 
abilities, motivation, and confidence. 

Five facts were mentioned by students to describe what they liked most about the E-Mail communication: 

(1) communicating with native speakers, (2) being exposed to native culture, (3) making friends, (4) practicing 
language, and (5) receiving responses efficiently. Four common facts were pointed out by students describing what 
they liked the least about the E-Mail communication: (1) unreliable system, 

(2) late responses, (3) different academic majors, and (4) limited time. Not being able to write with a Spanish 
accent, not learning oral skills, worrying about offending pen-pals, and disliking certain characters of some of the 
pen-pals were also mentioned by a few students. 

The instructor of the study learned E-Mail the same time as the students. However, because of his busy 
schedule, he could not communicate with his students via E-Mail. He mentioned that he himself had little 
experience in using E-Mail, but thought that E-Mail could be very useful in foreign language learning. Students 
could directly communicate with native speakers and discuss culture with them. He expressed that he would like to 
learn more about E-Mail and be familiar with it so that he could integrate it into his class in the near future. 
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Problems Encountered by Students 

The problems included the E-Mail system, different academic training, commitment of writing, access to 
computers, and limited knowledge. At the beginning of the study, the undeliverable mail caused problems for the 
students to communicate. The students also addressed difficulties in communicating with people with different 
academic majors. The Mexican students were encouraged to write to the U.S. students, but they were not required to 
do so. Therefore, some U.S. students had problems maintaining a dialogue with their pen-pals because they did not 
receive responses. In addition, limited access to computers prevented some students from writing to their pen-pals 
more often. Some students were not familiar with the use of E-Mail when the study started. Although the 
researcher helped some individuals after the initial E-Mail instruction, the researcher found that the students would 
have been less frustrated or that considerable time would have been saved if they had had more knowledge about E- 
Mail. 

Discussion 

No Difference Was Found in Comparison 

The results indicated that the foreign language learners in the E-Mail group did not perform foreign language 
better and did not have higher confidence than the students in the non-E-Mail group. The results of this study were 
consistent with other quantitative research where a significant difference in learning was not found (Mellgren, 1983; 
Williams, 1980; Barson, Frommer, & Schwartz, 1993; Boswell, Mocker, & Hamlin, 1968; Chu & Schramm, 
1967; Chute, Bruning, & Hulick, 1984; Hoyt & Frye, 1972; Kruh, 1983; Whittington, 1987). 

There is a trend to move away from comparison studies. Mclsaac and Gunawardena (in press) said, “It is 
time, therefore, to move away from media comparison studies that often yield no significant difference.” The 
questions that need to be asked “are not which medium works best, but rather how best to incorporate media 
attributes into the design of effective instruction for learning.” Following this trend, the qualitative data of the 
present study were further analyzed. The results indicated that E-Mail could motivate students, help in learning 
culture, enhance social presence, and assist foreign language learning. The participants were in favor of the use of E- 
Mail in foreign language learning. 

Motivation 

E-Mail motivated students. Throughout the semester, both groups received the same encouragement to 
conduct the out-of-class work to gain extra points. Nevertheless, the number of the students that participated was 
very different between these two groups. Sixteen out of 18 students in the E-Mail group conducted the work. None 
of the students in the non-E-Mail group did. The current research therefore suggested that E-Mail motivated the 
students. 

Culture 

E-Mail helped in learning culture. The E-Mail messages of the students were categorized into eight topics: 
culture, language accuracy, course management, social activity, books, class assignment, desire to communicate, and 
personal sharing. Like Cononelos and Oliva (1993) and Sayers (1994), the present study suggested that E-Mail 
contributed to cultural exchange. In this research, many cultural aspects were mentioned and discussed in the 
students’ E-Mail messages, including patterns of living, values, customs, beliefs, and manners. 

Via E-Mail, they talked about their classes, professors, universities, families, friends, jobs, and plans for 
the future. They discussed their favorite music and sports. Different singers, bands, and athletes were mentioned. 
Reading their E-Mail messages provided one with a very vivid picture of what a U.S. college student’s social life 
was like. Parties, “happy hour”, beers, clubs, dances, and dates were favorite topics of these students. These topics 
led to discussions about legal drinking age, appropriate age for marriage, and roles of genders in dating. Questions 
such as "Do people have to reach twenty-one years old to be able to drink?", "Is a man in his early twenties too 
young to get married?", and "Can a woman ask a man for a date?" were asked. 

The U.S. students shared how they felt about school. The majority of them mentioned a heavy workload. 
Some students complained how tiresome school was while others expressed enjoyment of what they were doing. 
They also talked about what they did on holidays, and how festivals were celebrated in their countries. 

In addition, the participants discussed social, political, and economic issues in their nations. They shared 
what they thought of their political leaders and current economic situations in their countries. Disasters, like the 
bombing in Oklahoma and tornadoes and earthquakes that occurred in Mexico, caused great concern to the 
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participants in the E-Mail communication. For these participants, the disasters were not only social events in a 
nation, but also meant damage or pain to their pen-pals, who they encountered in the E-Mail communication and 
who they became friends with afterwards. 

Social Presence 

Via E-Mail, several U.S. students asked their Mexican pen-pals about the damage caused by the disasters in 
Mexico and expressed their deep concern. Unlike the experimental researchers' claims, the U.S. students' E-Mail 
contained many friendly and warm messages. Moreover, one can notice their deep friendships and warm relations 
from the level of intimacy in the following students’ E-Mail messages. 

A male student told his female pen-pal that he was very sad and confused because he wondered whether his 
girlfriend loved him as much as he loved her. His deep sharing moved the female pen-pal, and she told him frankly 
that she sometimes also had similar feeling toward her boyfriend and worried about the relationship. She suggested 
that he not think too much about it and encouraged him to focus on his studies. In her E-Mail messages, she wrote 
many meaningful statements to cheer him up. 

A female student shared her own problems with her male pen-pal and asked for help. She told him that her 
boyfriend was young, shy, and distant from her. She was frustrated about the relationship and felt deeply hurt. The 
male pen-pal shared his thoughts about the relationship from a man’s points of view and gave her advice. In 
addition, several male students expressed how they felt when they were rejected by women and hoped to get support 
from each other. All the sharing mentioned above was very personal, and the E-Mail messages contained high levels 
of intimacy. 

According to Short et al. (1976) and Walther (1992), the fewer codes that are available in a medium, the less 
attention is paid by the user to the presence of other social participants. This statement was inconsistent with the 
finding of the current study. Although E-Mail consists of few codes, the participants in the present study were 
willing to share deep and personal feelings with their pen-pals and received much attention from their partners. Their 
candid sharing caused their pen-pals to be open and share more about themselves. For example, the female pen-pal 
mentioned above, frankly told her pen-pal that she was deeply moved by his sharing and that she actually had the 
same worry as he did. 

Friendships between the U.S. students and the Mexican participants of the present study developed during 
the 10-week E-Mail communication. At the end of the semester, several U.S. students asked their pen-pals to 
continue exchanging messages via E-Mail. During the interview, the majority of the U.S. subjects said that they 
would like to continue the E-Mail communication with their pen-pals even though the research was over. Some of 
them stressed that they definitely would continue writing to their pen-pals. Towards the end of the semester, one of 
the students had been discussing via E-Mail with her pen-pals about meeting each other in Mexico. 

Foreign Language Learning 

E-Mail assisted foreign language learning. The participants of the current study discussed language use with 
their pen-pals. Many E-Mail messages of the U.S. students in this research were concerned with the Spanish 
language. Some of the U.S. students asked their pen-pals to correct their errors, including spelling and grammar, 
which appeared in their E-Mail messages. They also asked questions about Spanish slang and colloquial 
expressions. 

Several U.S. students stated that the language which their Mexican pen-pals used in E-Mail was different 
from the language which appeared in their textbooks or was used in their class. The language they learned in class 
was very formal. By communicating with their Mexican pen-pals, they were exposed to authentic Spanish. The 
students expressed that this exposure helped them to use Spanish in a natural way. Moreover, writing to native 
speakers helped them to recognize whether or not native speakers comprehended their Spanish. If their pen-pals 
understood them, they felt more confident about their language ability than before. 

Use of E-Mail 

The participants were in favor of the use of E-Mail. Some students in the present study had good 
experience in the communication with their pen-pals. They had similar interests as their pen-pals, and they had 
many topics to talk about. They frequently exchanged E-Mail with their pen-pals and received replies quickly. 
Some students did not have such a good experience. They seldom received their pen-pals’ responses and did not have 
much to talk about with their pen-pals. 
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No matter whether the students’ personal E-Mail experiences were good or not, the majority of the students 
thought that E-Mail was a positive addition to Spanish 313 and suggested that E-Mail should be integrated into 
foreign language instruction. The students in the present research enjoyed using E-Mail. 

The responses of the participants indicated that users were in favor of using E-Mail and they thought that E- 
Mail would be beneficial for foreign language learning. It enabled the students to receive feedback from native 
speakers within a short time. Direct communication with native speakers allowed them to expose themselves to 
authentic language and foreign culture. The participants (the instructor and the students) of the present study 
suggested foreign language instructors to integrate E-Mail into foreign language instruction. 

Suggestions for Professional Practice 

The following suggestions are for professionals who are interested in integrating computer-mediated 
communication (CMC) technologies, especially E-Mail, into instruction and who are considering conducting a 
project in which students communicate with learners of another country. Based on the present study, the suggestions 
are geared toward foreign language instructors, but they can be easily applied to instructors teaching culture, 
international relations, international business, and other related fields. 

The majority of the U.S. participants of the present study were not familiar with the use of E-Mail. The 
researcher found that the students would have been less frustrated or that considerable time would have been saved if 
they had had more knowledge about E-Mail use. Therefore, prior to starting a project, instructors should spend 
enough time on students’ E-Mail training. 

The power of E-Mail lies in the opportunity for the user to inexpensively communicate with people in 
different countries within a short time. The user can communicate with a native speaker and expose himself to an 
authentic culture. This is why E-Mail could be a useful tool for foreign language learners. Since students can 
directly communicate with people in another country, instructors should teach the students to be open and sensitive 
toward the other culture. They also ought to guide the students to learn about the culture of their pen-pals and to 
avoid stereotypes so that there are no misunderstandings or hurt feelings. 

In addition, educators need to instruct students to express themselves appropriately and respectfully. They 
should help the students to understand and appreciate the opportunity to communicate with their pen-pals, especially 
with native speakers in a target language. It could happen that some pen-pals do not reply to the students’ E-Mail. 
The instructors should guide the students to properly express their hope to their pen-pals and to be patient. Of 
course, the instructor should also make arrangements for students whose pen-pals do not reply. Pushy and impolite 
statements should definitely be discouraged. Instructors should also be concerned about the E-Mail messages 
between the students and their pen-pals and should discourage the students from improper writing. 

Although some participants of the study addressed problems of having pen-pals with different academic 
majors, the researcher thought that communication between people with different majors could be interesting and 
rich. People with dissimilar academic training sometimes view things differently. Participants could expose 
themselves to viewpoints which are different from theirs. Of course, communicating with the students with very 
different majors requires the instructor to provide the students with good guidance. 

Currently, more science students than humanities students use E-Mail or technology in their fields. A 
humanities instructor therefore should be aware that it might be more difficult for him to find pen-pals with 
humanities majors than pen-pals with science majors to communicate with his students. The instructor also should 
be aware of the strengths and weaknesses of having humanities students communicate with pen-pals with science 
majors. The instructor should inform students ahead of time and prepare them for problems of communicating with 
the students with very different majors. 

If one uses E-Mail in class, integration of the technology into instruction is essential. An instructor must 
integrate technology into instruction, rather than just use it in class. The integration requires a good course design. 
The instructor should prepare the participants for the communication with their pen-pals at the beginning of the 
project. Providing the participants with appropriate questions to get to know their pen-pals is one way of making 
initial communication smooth. Another way is to collect enough background information of the pen-pals so that 
the participants can select their own pen-pals. Some other issues, such as respecting pen-pals’ culture, avoiding 
stereotypes, and following the etiquette of writing E-Mail messages, also need to be addressed with students. 

Once communication with pen-pals starts, guided questions which are related to content topics in class can 
be provided for the students to post to their pen-pals. For example, family structure might be one of the topics 
covered in a textbook and in class. The instructor should provide the students with questions such as "Who do you 
live with?’’; "When do people leave their parents to live by themselves?"; "Do people live with parents after they get 
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married?”; and "How do people feel when they move out of their parents’ house and live alone?" The instructor could 
prepare the questions or brainstorm the questions with the students in class. 

The instructor should encourage the students to find out if the answers of their pen-pals agree with what 
they learn in class. The content to be examined could include cultural aspects, language use, and other related topics. 
The instructor could ask the students to judge whether the culture described in their pen-pals’ E-Mail messages differs 
from what is mentioned in class. The instructor could also invite the students to compare their pen-pals’ writing 
with the writing in the textbooks. The educator should ask the students to present the information either by writing 
or orally. The instructor should allow enough time for students to share what they think of their pen-pals’ feedback 
and discuss agreement as well as disagreement in class. 

Since the project includes two nations, instructors at both sides of the project should be aware of how good 
the technical systems are in both countries. The technology is not as sophisticated in some countries as in others. 
The technical problems could cause delay of communication, even failure of communication; therefore the instructors 
should cooperate with each other to solve problems caused by technical systems. The present study could not have 
been completed without the assistance of the Mexican professor to solve problems. The instructors should inform 
the students about the possibility of technical problems ahead of time so that the students would not be too frustrated 
when problems do occur. 

In addition, the two instructors should work closely to solve communication problems among students. 
Some communication problems might be caused by limited technology, but some problems might be caused by 
individuals. For example, if a student does not receive a pen-pal’s reply, the server might be the problem. It might 
also be that the pen-pal was sick, that the pen-pal had dropped the class, or that simply the pen-pal was not interested 
in writing E-Mail. The instructors play an important role as mediators to provide students with accurate information 
about a pen-pal’s status and to assist E-Mail communication by either discussing the problem with the other 
instructor or changing the pen-pal. 

The instructor should also be aware that some students might select a pen-pal with a different gender or ask 
for changing a pen-pal with an intention of having a cyberspace date. The instructor should help the students to 
move their attention away from cyberspace romance and to focus on learning. As we mentioned, the instructor 
should be concerned about the students’ E-Mail messages. Once the instructor notices inappropriate or sexual 
remarks in the students’ messages, the instructor should step in to guide students. 

An instructor being involved in this kind of project is strongly suggested to learn the use of E-Mail. To 
solve the problems mentioned above requires an instructor’s ability to use E-Mail, unless the instructor has an 
assistant to take care of the problems. The instructor should communicate with the students via E-Mail because E- 
Mail might change the relationship between the instructor and the students. Currently, software allowing foreign 
characters in E-Mail is available. The instructor could use the software so that foreign language students have no 
problems to write foreign characters. 

The Follow-Up Study 

The follow-up study was conducted one year after the present study. E-Mail was sent to each of the 18 
students (the E-Mail group) who participated in the research. The students were asked to answer the following 
questions: (1) “Did you continue writing to your pen-pals after the research project stopped?”; 

(2) “Are you still communicating with your pen-pals via E-Mail?”; and (3) “How does the project affect you today?”. 
Four students answered the questions by E-Mail. Ten students answered the same questions through the phone, and 
four students could not be located. 

One third (four students) of the participants involved in the follow-up study said that they have continued 
writing to their pen-pals after the project was over. The communication lasted for a few weeks for two of the 
students and lasted for three months for one student. The fourth student was still communicating with her pen-pal. 
Other students did not continue writing to their pen-pals because they were too busy to write or because their pen- 
pals did not reply. 

All of the students responded that using E-Mail in foreign language learning was a great idea and should be 
integrated into instruction. Several students said that the project had positively affected them. Three students 
described exactly how the project influenced them. 

One of the students mentioned that although his experience with his pen-pals in Spanish 313 was "short 
lived", the project has motivated him to make friends with people in Brazil and has helped him to "remain in touch 
with the culture there." Another student stated that his pen-pals in Spanish 313 were not "very responsive.” 
However, he has found other pen-pals and wrote to them weekly. He said, "Your study got me interested in writing 
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to people, and I have found several lists of Spanish pen-pals on the Internet.” Another student mentioned that she 
was helping her daughter’s school to connect with people in Mexico. She wanted to utilize E-Mail as the study did. 

Conclusion 

Few studies have been conducted on E-Mail and foreign language learning. The present research was one of 
the few studies to investigate the impact of E-Mail on foreign language learning in which the foreign language 
learners communicated with native speakers in the target language for a long period of time. Both quantitative and 
qualitative data were collected for the current research. 

The results of the quantitative data indicated that the students who used E-Mail and the students who did not 
use E-Mail did not significantly differ in their language performance and confidence. According to the analysis of the 
qualitative data, E-Mail was found to be able to motivate students, help in learning culture, enhance social presence, 
and assist foreign language learning. The results also showed that both the instructor and the students were in favor 
of the use of E-Mail. Therefore, E-Mail could be a very useful tool for foreign language learning. 

During the study, technical and communication problems occurred. Fortunately, the problems were 
overcome, and the study was successfully conducted. It is suggested that instructors prepare students for E-Mail 
communication before the project started. They should teach the students to be open toward cultures and avoid 
stereotypes. E-Mail etiquette should be taught. Moreover, integration is an essential issue. It is strongly 
recommended that professionals integrate E-Mail into instruction, rather than just use it in class. A good course 
design is necessary for the integration. 

The present study was well perceived by the participants. The results of the study indicated that E-Mail 
may be beneficial for foreign language learning. The findings of the follow-up study indicated that the research 
project had a positive impact on the students over time. It is recommended that researchers expand the present study 
to further investigate the role of E-Mail in the relationship between instructor and students, re-examine the impact of 
E-Mail on foreign language learning or other settings, and investigate the relationship between CMC and social 
presence. 
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Appendix A 
Attitude Survey 

Name: ID#: 

Answer with one of the following: (1) Strongly disagree, (2) Somewhat disagree, (3) Neutral, (4) Somewhat agree, 
(5) Strongly agree. 

1. I feel confident speaking Spanish in my class. 

2. I feel confident about having casual conversations with native Spanish speakers in Spanish. 

3. I feel confident about having an academic discussion (i.e., culture, politics) with native Spanish 

speakers in Spanish. 

4. I feel confident about having written communication with native Spanish speakers in Spanish. 

5. My Spanish is good enough to travel around Spanish-speaking countries. 

6. My Spanish is good enough to study in Spanish-speaking countries. 

7. My understanding of the life style in Spanish-speaking countries is good. 

8. My understanding of values in Spanish-speaking countries is good. 

9. I learn foreign language, especially Spanish, fast and well. 

10. 1 am a good language learner in general. 



Appendix B 
Cloze Test 

From the answer key below, select the word that best fits the text and write the letter next to the corresponding 
number. 



(A) fiestas 


(B) padres 


(C) pafses 


(D)y 


(E) se 


(F) en 


(G) agruparse 


(H) familia 


(I) los 


(J) tambien 


(K) con 


(L) publicos 


(M) diferentes 


(N) de 


(O) la 


(P) una 


(Q)es 





Un gran numero de acontecimientos sociales son de tipo familiar. En los dfas de fiesta y los domingos las 
familias ffecuentemente se reunen en la casa de alguien o bien, en un restaurante de tipo familiar. Estas fiestas 
familiares se caracterizan por la presencia de los ninos y los abuelos. 

Algo que atrae la atencidn 1 los norteamericanos cuando visitan los 2 hispdnicos es la presencia de 

3 ninos en casi todas las 4 . Elios est£n acostumbrados a participar 5 los adultos en las fiestas 6 

en otros acontecimientos, como bodas 7 bautismos. Desde muy pequenos, participan 8 la vida social de la 

9 .. Asi aprenden continuamente c6mo comportarse 10 sociedad. Estdn acostumbrados a tratar 11 

personas de diferentes edades — abuelos, 12 y hermanos mayores — , desarrollando asi 13 actitud de respeto 

quemantienen 14 cuando son adultos. En lugares 15 , como el cine o los bailes, 16 ven grupos de 

personas de 17 edades. Hay menos tendencia a 18 segun la edad, como en 19 sociedad norteamericana. 

Por eso, tambidn 20 menos molesto llevar a la mamd o al hermano menor cuando dos jdvenes van al cine. 

I. 2. 3. 4. 5. 

6. 7. 8. 9. 10. 

II. 12. 13. 14. 15. 

16. 17. 18. 19. 20. 



O 
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Appendix C 

Questionnaire on E-Mail Use 
Name: ID#: 

Please answer the following questions frankly; this is confidential. Answer with one of the following: (1) Strongly 
disagree, (2) Somewhat disagree, (3) Neutral, (4) Somewhat agree, (5) Strongly agree. 

1. E-Mail has improved my ability to write in Spanish. 

2. E-Mail has improved my ability to read in Spanish. 

3. E-Mail has improved my ability to communicate in Spanish. 

4. E-Mail has enhanced my understanding of Spanish-speaking culture. 

5. E-Mail has increased my confidence about my Spanish ability. 

6. E-Mail has increased my motivation to continue learning Spanish. 

7. I learned a lot from my pen pal(s). 

8. I like to communicate with Spanish native speakers via E-Mail. 

9. E-Mail was a positive addition to Spanish 313. 

10. E-Mail should be integrated into Spanish language instruction. 

What did you like the most about E-Mail? 



What did you like the least? 



How do you think the use of E-Mail in your class could be improved? 



Language, Gender and Cyberspace: Pulling the Old Stereotypes into 

New Territory 
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Abstract 

The social construct of gender is laced throughout society and carries stereotypes through the use of 
language , arts ; literature , and social practices. Males and females do have things in common , but there are also male 
and female spaces that create divisions that are hand to cross. Ay more women begin using computers more 
frequently and in new ways , we see the stereotypes becoming evident in the patterns of communication in the 
cyberspace environment. 

Cultural Concepts and Communication 

Gender is a social construct and as such, it both influences and is a product of communication. Messages 
delivered to people over many years, through different social situations and various media, become part of the daily 
vocabulary that can perpetuate gender stereotypes. From a very early age, males and females are taught different 
linguistic practices. Communicative behaviors that are acceptable for boys, for example, may be considered 
completely inappropriate for girls. 

Research on women and language reveals that women experience linguistic discrimination in two ways: in 
the way they are taught to use language, and in the way general language usage treats them. So, for example, 
women reflect their role in the social order by adopting linguistic practices commonly associated with feminine 
mannerisms, such as using tag questions, qualifiers, and fillers to soften their messages, often making mild-sounding 
requests and apologizing for asking for things that are actual needs. Likewise, language directed at women were 
traditionally identified by their association with men, and occupational titles indicated which jobs were “for men” and 
which were “for women.” 

While much of this has changed today, our society retains a tendency to imply that maleness, after all, is 
the standard for normalcy, a female physician may still be referred to as a “woman doctor,” and while a female 
committee chair may be called the “chair” or the “chairperson,” a male in that role will more likely be called 
“chairman”. What is learned about gender, is reflected in language usage and graphical representations. 
Communicative practices not only reflect notions about gender, but they also create cultural concepts of gender. 

Cultural concepts of gender become ingrained within society to the point where they become accepted as 
“the way things are” or “the way things have always been.” Sometimes these concepts are so pervasive that they 
are barely noticeable because they are expected. Thus, we expect an investment counselor to speak in a certain way 
to women clients and in another way to male clients, based upon the language and level of knowledge perceived as 
important at the first meeting. This seems to be the case for computer consultants as well. Thus when a woman 
asks for consulting, she will be given different information than a male who asks the same question. Often, the 
consultants will “talk down” to women or give them less information or fail to the put the provided information 
into a meaningful context. 

The patterns of gender influence on communication carry into all areas of life, touching the home, the 
school, the community, the workplace, and so on. As more people become involved with computer-based 
communication, the patterns are evident there are well. 

Computer Culture and Cyberspace 

The existing male technology community was shaped by our society. Members of that community share 
characteristics in common which each other, and do not always know how to relate to others who are different from 
them. So, although they may not consciously try exclude women, they do not try include them either, thus the 
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gender gap in the computer culture and cyberspace widens as those in the mainstream gain more information while 
those not in the mainstream are omitted from the information pipeline. 

Cyberspace is not a gender neutral space. One of the characteristics of computer-mediated-communication 
(CMC) is its lack of easy social contextualization. CMC neutralizes such social status clues as appearance, voice, 
organizational hierarchy, and often gender, dependent, of course, upon the user ID, mailing address, and message 
content (Turkle, 1995). Despite the relative anonymity of the communication media, some women report that they 
are harassed and intimidated from posting and participating in on-line conferences or via e-mail. To avoid this some 
women choose gender-neutral user identifications (IDs), and prefer to post in women-only conferences or mailing 
lists (Turkle, 1995). 

Gladys We, of Simon Fraser University, investigated reactions of men and women to determine how they 
felt about on-line versus face-to-face communication (1993). She distributed a questionnaire to various UseNet 
newsgroups and an eclectic range of mailing lists in order to determine the answer to that question. She concludes 
that on the surface, it appears that most people believe that cyberspace tends to be friendly to women, allowing them 
to adopt more active personae, and to speak on a playing field that is more level due to reduced gender cues (We, 
1993). Yet, we need to look beyond that surface level assumption. After all, most people assume that women have 
an equal chance at becoming a scientist, a professor, a doctor, a lawyer, or the chief executive officer (CEO) of a 
large company. Yet all of these occupations are under represented by women and the subtle discriminations become 
obvious in the patterns of the information flow within the job. 

Messages and Myths 

Message sources privileged by society as legitimate knowledge generators create a web of socially 
compelling discourses. Thus, religious, mythic, philosophic, and scientific discourses teach us, among other 
things, about society’s values and rules related to gender. It is no accident, then, that American myths focus on the 
active male and the supporting female, or that Plato defined women as “lesser men,” or that Aristotle described 
women as “a deformity, a misbegotten male,” (Miles, 1989) or that craniologists of the nineteenth century argued 
that women’s smaller heads justified their subordinate position in society thus initiating all the “pretty little head” 
rhetoric about women (Tavris &1984), or that Freud believed women had “little sense of justice” (Osborne, 1979). 

The rhetorical force of myths in constructing powerful world views is awesome. There are no myths that 
are completely innocent or free of ideology, just as there are no “natural” myths. Every myth is a manufactured 
object, and it is the inherent bad faith of a myth to seem or to pretend to be a fact. Similarly, artistic representation 
has been especially potent. Since antiquity, women have been among the most popular subjects for painters and 
sculptors, most of whom have been male. Multiple roles have been assigned by male artists to women in art, both 
positive and negative. Women are depicted as objects of beauty, as symbols of passivity, as images of the 
powerless, and sometimes as images of power. All these privileged discourses create a web of meaning, a socially 
constructed world view that historically has excluded or made secondary the experience of certain groups of people. 

In addition, mass mediated messages offer the most contemporary, powerful, and technologically 
sophisticated strategies for shaping cultural reality. The beauty, diet, and advertising industries are the most 
obvious, best researched examples of contemporary, self-conscious myth-makers who control cultural concepts and 
acceptable images of gender of what it takes and means to be male or female, masculine or feminine (Wolf, 1991). 

Mass Media and Cyberspace 

The myriad of mass mediated communication forms available now range from the simplistic printing press 
to the information superhighway and beyond. The role of culture in communication practices directs us to an 
intercultural perspective on gender and communication. The social implications of computer mediated 
communication are vast, from its potential ability to overthrow centralized control of information to its potential 
ability to help people, no matter what their gender, race, or physical appearance communicate with each other with 
fewer prejudices and misunderstandings than any other medium in existence. In many ways, the on-line world, 
named “cyberspace” by William Gibson, has its own culture, morals, and expectations, but in many ways, it 
replicates the biases, contradictions, and prejudices of our society. 

Although the are increasing numbers of women who are actively using the cyberspace environment, it 
remains male dominated, and most training materials for the computer environment are written by males. As 
educators direct more students to on-line activities, develop more materials for World Wide Web (WWW) servers, and 
schools become involved with accessing and creating WWW pages, one would expect to find some guidance about 
issues of diversity on the WWW. Yet an extensive review of on-line WWW page design documents revealed that the 
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topic of gender representation in language and general page designs was not addressed by at all. One book does 
consider the importance of gender in graphic representations for the WWW, but it tends to sanitize the graphics rather 
than appreciate diversity (Horton, Taylor, Ignatcio, & Hoft, 1996). Horton, et al. recommend the following 
strategies for producing graphics for WWW pages that do not perpetuate sexual stereotypes and biased gender roles. 

1. Use simple , abstract figures, devoid of recognizable bone structure or hair style. 

2. Use unshaded line drawings of people. 

3. Use simple unisex cartoons or stick-figures drawings of people, hands, and faces. 

4. Simplify drawings of clothing to omit seams, folds, buttons, and belts. 

5. Do not show bare arms, shoulders, legs, or feet.. 

While this might be one way of avoiding gender stereotypes, it produces sanitized images and strips the 
medium of its potential graphic power. Reaching a balance of language and imagery in cyberspace will certainly 
need to find a neutral area in which the stereotypes come to rest while not sanitizing our culture to make us all the 
same. What good does it do to put a stick figure within a document that is loaded with biased language? Perhaps we 
need to look at some of the more subtle gender messages that are often overlooked in cyberspace and the computer 
culture in general. 
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Background 

There is a substantial research base considering the effects of headings within text, however there are almost 
no studies involving the use of headings during testing. Headings in text research results have been mixed. 
Headings in text appear to affect the encoding of certain kinds of information including declarative information and 
recall of concepts, but has been shown to be nominally effective for recall of logical relations. Also, lessons 
including headings seem to benefit less able readers, older readers, and field dependent readers. Learner-generated 
headings positively impact delayed recall of inference especially for higher-ability learners. 

The effects of headings during testing were investigated by Townsend, Moore, Tuck, and Wilton (1990). A 
sample of undergraduate students (n = 287) enrolled in an introductory educational psychology course received a 41 
item multiple-choice examination during the first quarter of the course that was 20% of their final course grade. The 
students were randomly assigned to receive either the version with headings or the version without headings. The 
headings were based on the title of the chapters covered during each part of the course. Field dependence-independence 
and student attitude towards the use of headings during testing were also considered. Headings did not facilitate (or 
hinder) test performance and no association was observed between field dependence-independence and headings. 
Interestingly, the students overwhelmingly reported that they felt that headings during testing would positively 
influence their performance. 

Aims 

Do headings within multiple-choice tests impact test performance? It has been suggested that headings 
within multiple-choice tests positively affect test performance, however, there is little research to date to either 

support or reject this hypothesis. This study (^Marcinkiewicz & Clariana, 1997) replicates and extends Townsend et 
al. (1990) by utilizing a different population and by including a covariate to increase statistical power. 

Sample 

The available sample for this study included employees of a large manufacturing plant in the western part of 
the United States (n=143). Participants ranged in age from 26 to 64 years old, the median age was 44 years old. 

Method 

Participants were randomly assigned to the experimental or control treatments. The experimental group 
received a multiple-choice recertification test with headings included; the control group received the same test without 
headings. The test consisted only of questions; no text passages were included. The multiple-choice test items were 
developed by a panel of subject-matter experts (SMEs) and test specialists. The items were pilot-tested and revised 
by a second group of SMEs. The final test version was approved by SMEs and plant management. The test 
included five content areas: Mathematics, Physics and Nuclear Physics, Radiation Biology and Exposure Control, 
Dosimetry, and Radiation Instrumentation. The KR-20 reliability was 0.79. 

The test items were grouped by content area. The words used as headings for each item grouping were the 
same as the content areas listed above. This type of heading is approximately equivalent to the use of chapter titles 
as headings in the Townsend et al. (1990) study. 

Results 

The data were analyzed by analysis of covariance (ANCOVA), a second similar test served as the covariate. 
A significant difference was obtained for the treatment main effect, F(l, 140) = 4.33 (p < .04), with a calculated 
effect size for headings (x = 76.4) over no headings (x = 74.7) of es = 0.33 (adjusted). A post hoc analysis of the 
influence of headings on test items of differing difficulty was conducted. This analysis indicates that headings in 
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tests have little effect on difficult test items, but a powerful effect on easier items (see Table 1). Table 1. Impact of 
headings by test item difficulty. 



item difficulty: 


difficult 


moderate 


easy 


No-headines^roup 


x = 


51.8 


73.5 


89.8 




s.d. 


(10.5) 


(8.7) 


(3.8) 


Headings group 


x = 


51.6 


77.3 * 


93.3 * 




s.d. 


(6.6) 


(7.4) 


(2.8) 


effect size 




0.02 


0.44 


0.92 



Note: * significant at p<0.05 level; No-headings group, n of subjects = 60, 
Headings group , n of subjects = 83 



Conclusions 

Contrary to a similar earlier study by Townsend et al. (1990), the findings of the present study indicate that 
headings within multiple-choice tests positively affect test performance. As in the Townsend study, the headings 
used in the present study apply at the "macro" level of the tested content. In both studies the tests were relatively 
important to the participants. Though the participants in both studies were adults, the populations were very 
different. Also, the use of ANCOVA in the present study allowed for increased power of discrimination compared to 
the earlier study. 

Perhaps most important, the findings of this study suggest that headings may differentially impact test 
items of differing difficulty. Thus, uncontrolled item difficulty effects may account for the lack of significance in the 
Townsend study. Further, previous mixed results for headings in text may be due to uncontrolled item difficulty 
effects. A meta-analysis of headings in text studies should consider item difficulty. Future headings within test (and 
text) research should control item difficulty. 

It is advised that selecting appropriate headings to use during testing is important. Headings that are highly 
specific to each item could reasonably be expected to obtain a greater effect than the generalized or "meta" level of 
headings used in this study. Future research should focus on increasing heading specificity. Possibly learner 
generated headings may provide the highest level of specificity with the greatest effect. 
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Independent Learners? 
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Abstract 

Students enrolled in courses offered through cm off-campus professional agriculture degree program were 
more likely to possess a field-independent cognitive style. Furthermore ; the agricultural distant learners were 
relatively more field-independent than the original GEFT norm group (Witkin et ai, 1971) and a college of 
agriculture norm group ( Torres , 1993). Both field-independent and field-dependent learners were satisfied with the 
distance delivery options provided through the off-campus professional agriculture degree program. Perhaps , the 
most notable difference between the field-independent and field-dependent learners was the more positive response of 
field-independent learners regarding the likelihood of enrolling in additional agriculture courses delivered through 
distance education technologies. Besides issues of diversity , knowledge of cognitive styles of agricultural distant 
learners may have implications for selecting instructional strategies. Theoretically, instruction that is in harmony 
with an individuals learning style will improve the student's performance, shorten study time, and improve the 
student's attitude toward learning (Chinien & Boutin, 1993). Further research is recommended to test the effect of 
style specific instruction in agricultural distance learning programs. In the interim, a variety of delivery tools, 
methods, and social support structures should be used. 

Introduction 

Cognitive theory concentrates on the conceptualization of students’ learning processes. It focuses on the 
way information is received, organized, and retained by the brain (Thompson, Simonson, & Hargrave, 1991). 
Knowledge of students' learning styles can aid educators in understanding these mental processes. DeBello (1990) 
defined learning style as "the characteristic cognitive, affective, and psychological behaviors that serve as relatively 
stable indicators of how learners perceive, interact with, and respond to the learning environment” (p. 203). Much 
of the learning style research (Cano, Garton, & Raven, 1992a, 1992b; Cano & Metzger, 1995; Garton, 1993; 
Torres, 1993) done by agricultural educators involved assessment of the field-dependence/ independence psychological 
dimension. This dimension relates to global vs. analytical perceiving and the ability to perceive items without 
being influenced by the surrounding field (Chinien & Boutin, 1993). 

Although a plethora of research exists on learning styles, with several published works related to 
agriculture, no published research was found that investigated the learning styles of agricultural distant learners. 
Distance education programs in agriculture are becoming more common, yet little is known about the characteristics 
of students who enroll in these programs. Research (Miller, 1995a; Miller & Honeyman, 1993) involving students 
enrolled in one agricultural distance learning program suggests that distant learners prefer being able to control the 
pace of their learning, prefer independent study, have less need for structured learning experiences, and have less need 
for interaction with the instructor and with other students. Garton (1993) provided a thorough review of the 
literature on field-dependent and field- independent learner preferences. This review emphasized the extremes of the 
continuum of learning styles, and not all learners of either learning style preference will necessarily exhibit all 
characteristics and behaviors associated with their style. Selected learner characteristics and behaviors from Garton’ s 
review are summarized in Table 1. When what is known about agricultural distant learners and distance learning 
programs is related to the learning style preferences identified by Garton, one might reasonably conclude that field- 
independent learners are better suited to distance learning programs than their field-dependent counterparts. Can such 
a conclusion be substantiated through research? Learning styles research in agricultural distance education is needed 
to promote our understanding of distance teaching and learning. 



Table 1. Selected Characteristics and Behaviors Associated with the Field-Dependent and Field-Independent 
Learning Styles 

Learning Style Field-Dependent Field-Independent 



Prefer externally defined goals and organization. 


Can provide their own structure for learning 
activities. 


Value positive reinforcement from the teacher. 


Do not typically respond to positive 


Have well-developed social skills and are more 


reinforcement offered by teachers. 


attuned to social cues. 


Have poorly developed social skills and are more 


Favor extrinsic motivation. 


socially independent. 


Prefer collaboration. 


Are intrinsically motivated. 




Prefer competition. 



Purpose and Objectives 

The purpose of this descriptive study was to determine whether field-independent learners are better suited to 
agricultural distance education programs than are their field-dependent counterparts. The research objectives were as 
follows: 

1 . Determine if students enrolled in agricultural courses delivered through distance education technologies were 

characterized more by the cognitive style of field-independence than normative groups. 

2. Compare attitudes of field-dependent and field-independent learners toward distance education delivery tools 

(i.e., videotape, interactive communications network [ICN]). 

3. Describe relationships between cognitive style and selected student characteristics. 

Methods 

The population for this descriptive correlational study consisted of all students seeking a degree and/or 
advanced formal training in agriculture through an off-campus professional agriculture degree program at a 
Midwestern land-grant university. The sample (n=191) included all students who enrolled in one or more of the eight 
agricultural courses delivered through distance education technologies during Spring and Fall Semesters of 1995. 
Courses were offered in agronomy (three courses), agricultural systems technology, animal science, animal ecology, 
sociology, and biochemistry. 

The Group Embedded Figures Test (GEFT) (Witkin, Oltman, Raskin, & Karp, 1971) was used to determine 
the preferred cognitive style of the distant learners. The GEFT is a standardized instrument with a reliability estimate 
of .82. Also, concurrent validity with the Embedded Figures test was .82 for males and .63 for females. 

To compare attitudes of field-dependent and field-independent learners toward videotape and ICN-delivered 
instruction, a median split was used (Spanier & Tate, 1988; Thompson & Knox, 1987). There is no widely agreed 
upon score that precisely separates field-dependent and field-independent learners. It is correct, however, to conclude 
that within a group of students those who score above the group median on the GEFT are relatively more field- 
independent than those who score below the group median. In this study students who scored below the group 
median of 14 on the GEFT were labeled field-dependent, and those with scores equal to or greater than the median 
were labeled field-independent. 

The scale for assessing attitudes toward distance delivery media consisted of 1 1 Likert-type items with 
response categories ranging from strongly disagree (1) to strongly agree (5). The instrument was previously 
developed and validated by Miller and Honeyman (1993). The Cronbach's alpha reliability coefficient for the 
attitudinal instrument was .86. 

Identical data collection procedures were used for both Spring and Fall Semesters. Attitudinal and 
demographic data were collected by mailed questionnaire. One complete follow-up of nonrespondents was conducted. 
One hundred and sixty students (83.8%) completed and returned the questionnaire. The GEFT was administered by 



proctors during a regularly scheduled examination. A letter was sent to all students included in the sample (n=191) 
approximately one week before the GEFT administration to explain the purpose of the study and to encourage their 
participation. One hundred (52.4%) students completed the GEFT and 89 (46.6%) students completed the GEFT and 
the questionnaire. 

All data were analyzed with the SPSS personal computer program. Appropriate statistics for description 
(frequencies, percentages, means, and standard deviations, Pearson correlations, point biserial correlation, and 
Cramer's V statistic) were used. The magnitude of all relationships was interpreted by using Davis’ (1971) 
descriptors. 

Results 

The mean score on the GEFT was 12.38 for females and 12.53 for males with a standard deviation of 4.41 
and 4.46, respectively. GEFT scores from the present study were compared with normative groups to determine 
whether agricultural distant learners tended toward a field-independent cognitive style. Both male and female students 
in the present study obtained scores on the GEFT that were higher than the original normative group reported by 
Witkin et al. (1971) in the GEFT manual. Higher scores on the GEFT are associated with the field-independent 
cognitive style. Also, mean scores from the present study were compared with those reported by Torres (1993). 
Torres studied a representative sample of senior college of agriculture students. Females in the present study 
obtained higher GEFT scores whereas males attained lower scores than those in the Torres study (Table 2). 

Table 2. Comparison of the Mean Scores on the GEFT for Male and Female Subjects in the Present Study with 
Scores of Normative Groups 





Present Study 


Witkin et al, 1971 


Torres, 1993 


Females 


n 


21 


242 


44 


Mean 


12.38 


10.8 


11.1 


S.D. 


4.41 


4.2 


4.6 


Males 


n 


79 


155 


59 


Mean 


12.53 


12.0 


13.4 


S.D. 


4.46 


4.1 


3.8 


Overall 


n 


100 


397 


103 


mean 


12.50 


11.27 


12.40 


SD 


4.43 


4.2 


4.27 



Torres (1993) classified students who obtained GEFT scores above the national mean (11.3) as field- 
independent. Using this classification procedure, Torres labeled 61.2% of the college of agriculture seniors as field- 
independent. The same procedure, when applied to the current study, classified 62% of the students as field- 
independent. 

Attitudes held by field-independent and field-dependent students toward instruction delivered by distance 
education technologies (i.e., videotape, ICN) were compared. Seventy-nine students provided attitude toward 
videotaped instruction and cognitive style data. Thirty-seven students were categorized as field-dependent and 42 were 
labeled field-independent. Overall, both groups held positive attitudes toward videotaped instruction and were 
particularly positive about the opportunity for learning, the convenience, the ability to control the learning pace, and 
the prospect of enrolling in additional videotaped courses. Field-independent learners were only slightly more 
positive about videotaped instruction than field-dependent learners. One notable difference was that field-independent 
learners were more positive about the likelihood of enrolling in another videotaped course (Table 3). 

Twenty-three students provided attitude toward ICN and cognitive style data. Thirteen students were 
categorized as field-dependent and 10 were labeled field-independent. ICN-delivered instruction provides a television- 
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based delivery medium via fiber optic cable with live two-way interactive audio and video. Field-dependent and field- 
independent students held positive attitudes toward ICN-delivered instruction. Field-dependent learners were only 
slightly more positive than their field-independent counterparts. Field-dependent students provided greater mean 
scores on attitude items related to the ability to control the pace of learning and their preference for ICN over 
traditional classroom instruction. On the other hand, field-independent learners were less likely to feel isolated, more 
positive about the learning opportunity provided by ICN, and more positive about the idea of enrolling in another 
ICN course (Table 3). 



Table 3. Analysis of Mean Attitude Scores by Cognitive Style 







Videotape (n=79) 


ICN (n=23) 


Item 




~Xfd(n=37) 


Xfi (n=42) 


Xfd( n=13) 


Xfi (n=10) 


1 . 


I enjoyed learning from the videotaped/ 
ICN lessons. 


4.21 


4.40 


4.23 


4.20 


2. 


Videotape/ICN should be utilized more 
often to deliver agriculture-related 
instruction. 


4.22 


4.24 


4.31 


4.50 


3. 


I feel more isolated as a student when I 
take courses by videotape/ICN. 


2.89* 


2.90* 


3.31* 


3.60* 


4. 


I would recommend videotape/ICN 
courses to my friends. 


4.16 


4.35 


4.46 


4.20 


5. 


Learning through videotape/ICN 
instruction is convenient. 


4.55 


4.64 


4.38 


4.20 


6. 


Videotape/ICN allows me to control 
the pace of my learning. 


4.34 


4.38 


3.92 


3.40 


7. 


I prefer videotape/ICN to traditional 
classroom instruction. 


3.32 


3.38 


3.53 


2.70 


8. 


Learning through videotape/ICN is 
boring. 


3.79* 


3.71* 


4.15* 


4.00* 


9. 


I would enroll in another videotape/ 
ICN course. 


4.37 


4.62 


4.46 


4.80 


10. 


Videotape/ICN provides me with 
learning opportunities that I otherwise 
would not have. 


4.61 


4.64 


4.39 


4.60 


11. 


I would not take videotape/ ICN 
courses if I had some other means of 
acquiring course credit. 


3.37* 


3.48* 


3.69* 


3.90* 




Videotape total 
ICN total 


3.99 


4.07 


4.08 


4.01 



* Indicates negatively worded items that were reverse coded. Note:/J=field-dependent;/z=field-independent 
Note: Based on Scale: 1 = strongly disagree; 2 = disagree; 3 = undecided; 4 = agree; 5 = strongly agree. 



Negligible associations were found between cognitive style and gender, and attitude toward videotape delivery. 
Low positive associations existed between cognitive style and occupation and motivation to enroll. Low negative 
associations existed between cognitive style and age and attitude toward ICN delivery. Field-independent learners 
were somewhat more likely to have occupations related to fanning and agribusiness, whereas field-dependent learners 
were more apt to have occupations outside agriculture. Field-dependent learners were more likely to be motivated to 
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enroll in a distance course to pursue a degree, whereas field-independent learners were more likely to cite personal or 
career development as motivating factors (Table 4). 



Table 4. Relationships Between Cognitive Style and Selected Student Variables 



Variable n 


Association 




Age 


100 


-.11“ 


Gender 


100 


-,01 b 


Occupation 


97 


.15 c 


Motivation to enroll 


96 


.13 c 


Attitude toward videotape delivery 


79 


.09“ 


Attitude toward ICN delivery 


23 


-.14“ 



Note: ^Pearson; b =point biserial; c =Cramer's V 



Conclusions and Recommendations 

As a group, the agricultural distant learners studied were relatively more field-independent than the original 
GEFT norm group (Witkin et al., 1971) and a College of Agriculture norm group (Torres, 1993). Additionally, 
field-independent learners were more positive about the likelihood of enrolling in additional courses delivered by 
either videotape or ICN. Perhaps field-independent learners are somewhat better suited to this agricultural distance 
learning program. The proportion of field-dependent learners enrolled in agricultural distance education programs 
should be routinely monitored. These data should be shared with faculty and administrators to promote awareness of 
the characteristics of learners enrolled in their programs. Although the distribution of cognitive styles was not 
drastically skewed in the current study, efforts to encourage diversity of cognitive style in agricultural distance 
learning programs may be warranted. 

Although the orientation of this group, particularly the female students, to a more field-independent 
cognitive style was noted, field-dependent learners were equally satisfied with videotape delivery of instruction when 
compared with their field-independent counterparts. Perhaps the range of student services and support mechanisms 
provided through the off-campus agriculture degree program has helped to create a climate that satisfies many 
preferences attributed to the field-dependent learner. The off-campus program provides an academic advisor to assist 
students in planning their programs of study. The advisor also assists students with registration and acquisition of 
course materials and often serves as a liaison between the student and the instructor. Besides advising support, 
students are provided access to their instructor(s) through a toll-free phone number. Also, Instructors with off- 
campus teaching experience are becoming more adept at identifying and meeting needs of distant learners through a 
variety of mechanisms including the Internet. 

Field-dependent learners were slightly more satisfied with ICN overall, but the difference between their 
attitude score and the one provided by field-independent learners was less than was expected. A much larger difference 
was found by Miller (1995b), but Miller’s study involved a sample only one-third as large as this study. ICN 
provides more opportunities for interaction with the instructor and other students than does videotape, yet requires 
that students sacrifice some of their control over the learning environment. This environment, it seems, would be 
relatively more appealing to field-dependent learners. Learners from both cognitive style groups are satisfied with the 
distance learning options provided through this program, but for different reasons. When the item means for the 
attitude scale are considered, the scores are reasonably compatible with what we know about field-dependent and field- 
independent learners. 

Besides issues of diversity, knowledge of cognitive styles of agricultural distant learners may have 
implications for selecting instructional strategies. Theoretically, instruction that is in harmony with an individual's 
learning style will improve the student's performance, shorten study time, and improve the student's attitude toward 
learning (Chinien & Boutin, 1993). Van Vuren (1994) concluded that more positive attitudes and improved academic 
achievement are promoted when instructional methods account for learning style preferences. Van Vuren, however, 
did not focus on the field-dependence/ independence psychological dimension commonly studied in agriculture. 
Furthermore, Claxton and Murrell (1987) report that the small amount of research involving this dimension is 
contradictory. Further research in agricultural distance learning should seek to isolate cognitive styles, design style 





specific instruction, and test the effect on achievement and satisfaction. Meanwhile, students should be given 
distance learning options various degrees of structure, interaction, and control of the learning environment. No one 
delivery tool or method will satisfy all learners. Instead, one tool or method used exclusively will discriminate 
against groups of learners. 

Summary 

Logic might suggest that distance education by its very nature is best suited to learners with a field- 
independent cognitive style. This study, however, suggests that distance learning programs can be developed to meet 
the needs of both field-independent and field-dependent learners. The key to success for field-dependent learners is 
likely related to the support provided by the off-campus programs office and the innovative methods used by 
instructors to enhance student-content interaction, student-student interaction, and student- teacher interaction. 

Is an understanding of cognitive style important for enhancing teaching and learning? Many would answer 
this question yes. In fact, Torres and Cano (1995) recommended that workshops be conducted to help instructors, 
students, academic advisors, and counselors become knowledgeable about learning styles. Such workshops may be 
helpful in that an elevated awareness of learner diversity and a renewed focus on learning might be achieved. A key 
question remains, however. What specific instructional interventions can be made to help learners with different 
cognitive styles experience success in their courses? College teachers of agriculture should engage in action research 
to find practical ways of using learning styles data to improve instruction. 
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Introduction 

How teachers plan instruction is a topic of interest to teacher educators as well as to many instructional 
designers. Prescriptions for planning based upon an objectives-first, linear or rational model are still advanced in 
teacher preparation programs (Borko & Shavelson, 1990), and there is some indication that teachers who are 
knowledgeable of basic principles of instructional design do, at least in their mental planning, employ important 
elements of generic models (Applefield, 1992; Earle, 1992; Clemente & Martin, 1990). Other studies indicate that 
while experienced teachers do not seem to follow the step-by-step objectives-first approach in their thinking and 
planning, the components of this approach and instructional systems design models are present in the experienced 
teachers’ processes of thinking about teaching during preactive, interactive and postactive thinking (Moallem, 
Driscoll, Papagiannis, & Strazulla, 1994; Moallem, 1993). 

However, there is a considerable body of data that fails to support the contention that teachers actually 
engage in a systematic process of planning (Borko & Niles, 1987; Egeler, 1992; Sardo-Brown, 1990). Experienced 
teachers typically do not plan and provide instruction in accordance with the objective-first procedures taught in 
teacher education programs (e.g., Shavelson, 1983; Brown, 1988) and advocated in instructional design models 
(Gagne, 1995; Dick & Reiser, 1996; Dick & Carey, 1990). As teachers talk about their instructional planning, 
what is often revealed is a focus on instructional content and activities, rather than on designating objectives, 
determining matching instructional activities and planning congruent evaluation (Berliner, 1990; Clark & Peterson, 
1986). 

Some researchers have found that preservice teachers who received instructional design training approached 
planning in a manner which was systematic and quite consistent with ISD models and principles (Earle, 1992; 
Reiser, 1994). Other studies demonstrated that preservice teachers who received instructional design training used 
language consistent with systematic models when asked to describe the planning process (Applefield, 1992; Driscoll, 
Klein, & Sherman, 1994). And even though there were some differences between teachers and instructional design 
experts in interpretation of instructional events (there were also differences among the experts), the study of 
traditional principles of instructional design did result in changes in teachers’ perspectives about the teaching-learning 
process. 

While these findings are encouraging, there is insufficient evidence to determine whether preservice teachers 
will continue using a systematic planning approach after they have completed their study of subject matter 
pedagogical courses and begin their student teaching. Certainly one factor that may account for failure to apply a 
systems approach in planning is the degree to which preservice teachers are grounded in a systematic approach to 
designing instruction. Other factors that influence the planning practices of teachers include textbooks (goals), 
teacher manuals and other curriculum materials, physical facilities and students, including their ability, gender, social 
competence, self-confidence and class participation (Brophy, 1982; Shavelson & Stem, 1981). Also reported in the 
literature are school schedules and a variety of organizational factors such as system or school goals, principals' 
planning requirements, class size and grouping practices (McCutcheon, 1980; Sardo-Brown, 1988). 

In addition, individual differences in teacher beliefs, educational philosophy and pedagogical training are all 
likely to play important roles. It should be noted that cited discrepancies between teacher planning practices and 
systematic instructional planning grounded in objectives-first models are paralleled by the ongoing debate within the 
instructional design community concerning constructivist perspectives versus traditional objectivist views on the 
nature of and procedures for designing effective instruction (Jonassen, 1991; Dick, 1995; Willis, 1995; Cennamo, 
Abell, & Chung, 1996). Challenges to objectivist ID views also find support in teacher education where the 
pedagogy taught over the past 10 years has increasingly moved to advocate constructivist approaches to teaching and 
learning (Goodman and Goodman, 1990; Brooks & Brooks, 1993; Confrey, 1990). Teachers' beliefs about learning 
and teaching will understandably be influenced by their methods courses, and constructivist-based approaches to 
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pedagogy and instructional design do not favor an objectives-first approach to teaching and planning. Instead they 
advocate recursive and reflective models for planning (Willis, 1995). 

Purpose of the Study 

The purpose of the study was to describe the development and explore the processes of thinking, planning 
and decision making of two preservice teachers as they progressed through their teacher education program. The 
effects of training in the systems approach on two preservice teachers' thinking about instruction and on their actual 
instructional planning documents were examined. Also investigated was the relationship between these teachers’ 
training in instructional systems design and the approaches to teaching and planning for instruction presented in the 
pedagogical courses of their teacher education program. The following questions were addressed: 

• What do preservice teachers think about learners, learning and teaching and how does their thinking change 
over time and after receiving ISD training? 

• What do preservice teachers think about instructional systems design and how does their thinking change over 
the course of their teacher education program? 

• How do preservice teachers who are trained in ISD plan their instruction, and how do they carry it out during 
their student teaching? 

Methodology 

Setting 

In this longitudinal study we tracked two preservice teachers as they proceeded through all stages of their 
teacher education program at the University of North Carolina at Wilmington (“UNCW”). The undergraduate teacher 
education program at UNCW requires students to complete a two-semester sequence course with the first course 
emphasizing instructional design and the second course focusing on classroom evaluation. The courses are designed 
to provide students with basic concepts and principles of instructional design and with the opportunity to practice a 
systematic approach to planning and evaluating instruction. Although major steps in generic instructional design 
models (namely, Gagne & Briggs, 1979 and Dick & Reiser, 1989) are used as the theoretical framework in both 
courses, the application of instructional design for teachers (the concept of "teachers as designers of their own 
courses") is being emphasized. In other words, throughout the two courses, students use a predictable sequence of 
generic instructional design steps to develop their instructional plans and make assessment and evaluation decisions. 

The instructional design component has been a unique characteristic of the teacher education program at 
UNCW. The inclusion of the ISD component into the program (Earle, 1985) has not been limited to taking the 
above-mentioned courses. The application of instructional design skills and principles is purportedly supported by 
the entire program in later methods courses and in the student teaching internship. For example, instructors of the 
methods courses and supervisors of student teaching have students develop lesson plans before any micro-teaching or 
independent student teaching. The program is also designed in a way that requires the instructional design course (the 
first course in the two- semester sequence courses) to be a prerequisite for most of the pedagogical courses. An 
objectives-first orientation to developing lesson plans is also implicitly agreed upon by the methods professors. 
However, despite the above suggested approaches in the program, the degree to which these programmatic supports 
are actually maintained throughout the program is uncertain. 

Participants 

Two preservice teachers who were enrolled in the instructional design course (the first course in the two- 
semester course) taught by the primary researcher participated in this study. The students were solicited on a 
volunteer basis with an understanding that they would make a three-year commitment to the research project. Neither 
of the volunteers had received any formal training in the use of systems design approach before entering the program, 
although they both had completed 45 credit hours of their basic studies. 

Both students (Chloe and Tammy, pseudonyms) are female, elementary education majors with no prior 
teaching experience and are considered traditional students. Both Chloe and Tammy took the two-semester sequence 
courses in instructional design and evaluation during the first year of their respective teacher education programs. 
Tammy took the two-semester sequence courses with the primary researcher, while Chloe took only one of the two 
sequence-courses with the primary researcher. Also, Chloe was supervised by the co-researcher in this study during 
her student teaching. Both students were considered average students with respect to their academic background and 



indicated that they had chosen to be in the teacher education program due to their love for teaching. Chloe completed 
her program of study in a slightly different order than Tammy. 

Data Collection Strategies 

The study employed a naturalistic approach and used several data sources. Data were collected over the 
three-year period beginning Fall, 1994 and continuing through Spring, 1996. The primary data sources were: 
unstructured interviews; students' notes, journals, assignments, projects and lesson plans; an instructional design 
survey; observations of student teaching; and personal communication with students' university supervisors. 

Data Analysis 

Several strategies were used to analyze the data. The general strategy was to analyze each source of the data 
separately before triangulating different sources to derive final assertions. This approach seemed appropriate since 
different data required different forms of analysis to be reduced for further analysis and comparison. 

The unstructured interviews, journals and other written assignments were analyzed using an open-coding 
approach (Strauss, 1987), which permits the categories to emerge from the data. Interviews and written assignments 
for each semester were read independently, and the key statements and phrases that gave expression in any way to the 
respondent's key ideas, beliefs and/or understanding of the learner, learning, teaching and teaching strategies, 
instructional planning, evaluation of instruction, or motivation were extracted. These statements were further 
classified by their content and were assigned a name. The categories and their related statements were then reorganized 
into a matrix to allow the researchers to examine each respondent's ideas and beliefs in the different semesters. The 
developed matrices with participants' phrases and statements were analyzed again using guiding codes derived from 
instructional design to further identify the participants’ thoughts about instructional design principles and procedures. 

Lesson plans and observation field notes were also analyzed using guiding codes derived from instructional 
design procedures and principles. Two lists of guiding codes were developed. One list was a summary of generic 
instructional systems design steps as prescribed by ISD models and taught in the instructional design sequence 
courses. Another list was generated based upon the major components of instructional design assuming an interactive 
relationship among the components. In other words, one guiding list assumed a procedural and step-by-step model 
for planning (See Table 1), while the other assumed interactive relationships among components without any 
preference for order of occurrence of each component. Recent interactive models of instructional design (Cennamo, 
Abell, & Chung, 1996; Willis, 1995) were used to generate the second list (See Table 2). The results of the 
analysis for the above-mentioned sources of data were organized into matrices that permitted detection of differences 
and changes in perspectives of each participant over time. 

The instructional design survey was analyzed using the keywords used by Driscoll, Klein & Sherman 
(1994) for items: 1, 2, 6, 7, 8, 10 and 11. For items where no keywords were identified (3, 4, 5, 9, 12, 13, 14), we 
used the instructional design guiding codes to assign the proper keyword. Items 9 and 13 were omitted due to the 
ambiguity of the items and the fact that no specific keywords could be assigned to these items. We also conducted 
quantitative analyses similar to that of Driscoll, et al., using the following steps: First, each participant’s response 
was judged either "not relevant" = 0, "relevant" = 1" or "very relevant" = 2 by each researcher separately. We judged 
the response as being "very relevant" if the participant used all of the identified keywords in her response; as 
"relevant" when part of the keywords were used or the keyword was implied; and "not relevant" when no keyword 
was used and/or the response did not address instructional design principles and procedures. Secondly, we compared 
our scores and upon agreement on each item, total scores were used to compare participants’ responses with each 
other and over time. 

Results 

What do preservice teachers think about learners, learning and teaching and how does their 
thinking change over time and after receiving ISD training? 

Chloe’s Beliefs and Dispositions: (See Table 3) By the end of her student teaching seminar at the beginning 
of Fall 1995, Chloe's concern for classroom management issues is quite apparent. She had formulated a specific set 
of strategies that concentrate on classroom management and she was eager to implement some of these strategies 
during her student teaching. Chloe’s early belief statements about making learning a fun activity for students and her 
desire to meet students' needs carried over into her student teaching. Chloe also states a strong belief and desire to 





incorporate whole language and cooperative learning methods, as well as to collaborate with parents. Although 
Chloe's elaborated thinking about teaching and learning point to the impact of the content of her education courses, 
her philosophy statements do not change materially. She uses many of the same terms and phrases that she had used 
at the beginning of the program to describe her beliefs about teaching and learning. 

Tammy’s Beliefs and Dispositions: (See Table 4) Tammy's entry beliefs and dispositions about learners, 
learning and teaching seem to be influenced by what she learned from her education courses. Tammy's entry concerns 
for differences among learners focuses most of her attention on distinguishing characteristics presented by her learners 
and relatively little analysis of how to teach her students. Her recall of personal early childhood experiences and her 
sensivity to the issue of discipline also seem to have been heightened by what she learned from the Individualized 
Instruction course. It seems that Tammy began her student teaching with strong beliefs about the importance of 
learners' individual differences, the value of an open classroom organization and a a personal preference for a 
somewhat laizzez faire classroom management approach and a flexible teaching style. Again, as in Chloe's 
case, Tammy's exit belief statements about learners, learning and teaching seem to be very similar to her entry belief 
statements; however, they have become more definite and specific and now appear to be expressed with the 
conviction of internalized personal knowledge instead of merely a set of abstract belief statements. 

What do preservice teachers think about ISD? and How has their thinking changed over the 
course of their teacher education program? (See Tables 5 and 6) 

Twelve questions were used for analysis (items 9 and 13 were eliminated) with a maximum possible score 
of 24, or 2 per question. Tammy scored 19 at the end of the fall semester prior to student teaching and 18 at the end 
of spring following her student teaching, missing only one item. Chloe’s overall score was lower than Tammy. She 
scored 17 at the end of Fall and 13 at the end of Spring missing two items at the end of Fall and 4 items at the end 
of spring. When we used the items and the scoring procedures (1 point for each correct item) that Driscoll and her 
colleagues (1994) used to score the participants in their study, Tammy scored 7 (out of possible 7) and Chloe scored 
5, missing items 6 and 1 1 at the end of Fall and items 7 and 1 1 at the end of Spring. A closer examination of survey 
items and the identified keywords showed an emphasis on motivation issues in the survey (3 items were devoted to 
motivation). Since other sources of data indicated that Tammy had a stronger concern for students' motivation, this 
emphasis could have masked other instructional design principles. After eliminating items related to motivation 
(items 5 and 10), Tammy scored 16 (out of a possible score of 20) at the end of Fall and 15 at the end of Spring. 
Chloe's scores were 13 and 9 respectively. Thus Tammy's responses remained relatively more consistent with ISD 
principles than did Chloe’s. 

A qualitative analysis of the participants’ responses to the statements revealed more meaningful 
explanations. Statement 7 and 12 emphasized the concept of "objectives" and the importance of informing learners of 
the learning expectations. Both Tammy's and Chloe's responses were relevant to the statement 7 and very relevant to 
statement 12 at the end of Fall 1994. In their responses to item 7, both Tammy and Chloe clearly indicated that the 
teacher should have informed students of her expectations although neither of them used the term "objective.” 
Chloe's response to the item 12 includes the term objective, but Tammy still does not use this term. However, both 
participants expressed clear thinking about objectives in this item. 

Comparison of each participant's responses at the end of Fall and Spring indicates a change. While Chloe's 
responses to items 7 and 12 are relevant and very relevant to the key concept at the end of Fall, her responses to the 
same items were judged to be irrelevant at the end of Spring. At the end of Spring, Chloe failed to indicate 
objectives as the main cause for the problem expressed in statement 7. Instead she pointed to the importance of 
providing an outline or summary of the content at the beginning of the lesson. She also seemed to follow the same 
trend of thought for item 12. Examining other sources of data provides a possible explanation for this change. First, 
Chloe began taking her methods courses which had more emphasis on the content of the lesson than its objectives. 
Secondly, Chloe’s assignments for the assessment course (the second course in the two-sequence course in ISD) 
showed that Chloe was instructed to use her beliefs and classroom instruction as references for making assessment 
decisions instead of using objectives as a reference for determining assessment strategies. 

As indicated above, Tammy's response for item 7 and 12 are relevant at the end of Fall, although she does 
not use the term objective. Her responses to the same items change from Fall to the Spring semester. At the end of 
Spring, Tammy clearly uses the term objective for item 7, which indicates stronger thought about ISD, but her 
response to item 12 is somewhat similar to Chloe's response. In her response she refers to the importance of 
communicating content outline instead of objectives, but she also indicates the importance of assessing students' 
learning during the lesson. Tammy's emphasis on the term objective at the end of Spring could be due to her 
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continuing use of the concept of objectives for making assessment decisions. Tammy's concentration on assessing 
students’ is consistent with the focus of instruction in her assessment course. 

Items 3 and 6 in the survey were targeted toward the concept of alignment between tests or assessment 
strategies and objectives. Chloe's response to item at the end of the Fall does not indicate the importance of the 
congruency between test items and objectives. But she points to the alignment between and testing in item 3. At 
the end of Spring, however, Chloe's responses are more relevant to both items, in each case indicating alignment 
between concepts taught and testing. This change could be due to the more elaborated discussion and information that 
she received on the assessment of instruction during Spring, 1995. 

Tammy's responses to items 3 and 6 at the end of the Fall address the problem of alignment between 
instruction and testing although she does not specifically point to objectives. At the end of Spring, Tammy not 
only points to the alignment problem but she also questions the process of construction of the test items. Again, the 
change in Tammy's knowledge could be due to the assessment of instruction course she had taken in Spring. 

Statements 2 and 11 addressed the principle of practice and feedback. Tammy's responses to these items 
were appropriate in both semesters. Chloe's response to item 2 is very relevant to the principles, but her response to 
item 1 1 is irrelevant in both semesters. Chloe's response to item 1 1 seemed to focus on the first part of the problem 
statement ("All she ever does is lecture") and as a result tended to see lack of variety of teaching methods as the 
probable cause of the problem. 

Statement 1 in the survey targeted one of the important principles in instructional design: prerequisites and 
remediation. Both Tammy and Chloe addressed the importance of the prerequisite knowledge in their responses at the 
end of Fall. Chloe's response at the end of Spring was similar to her response in the Fall; however, Tammy's 
response was slightly different. Tammy mentioned both prerequisites and remediation at the end of Fall but only 
mentioned remediation at the end of Spring. It should also be noted that Tammy and Chloe differed in their 
perception of prerequisite knowledge. Tammy seemed to emphasize the assessment of the previous lesson as a 
prerequisite for the new lesson, while Chloe saw the assessment of the prerequisite knowledge as a part of the 
procedure in the new lesson. Although this difference does not seem to be important with respect to instructional 
design procedures, it may indicate the difficulty of separating pre- and posts-lesson evaluation in instructional design 
models. The difference between Tammy's and Chloe's responses to this item could also be due to differences in their 
perception of teaching. As it was indicated in the previous section, Tammy is more in favor of a flexible approach to 
teaching, while Chloe seems to favor a more structured approach to teaching. Therefore, evaluation of the 
effectiveness of the previous lesson is more important for Tammy. Chloe, on the other hand, places importance 
upon the structure of each lesson and values beginning each lesson with assessment of the prior knowledge. 

Statements 4 and 8 required the participants to indicate the principle of example and demonstration or 
explanation. Taking into account the ambiguity of the statements for these concepts, we considered both Tammy's 
and Chloe's responses to these items relevant at the end of Fall. Tammy's responses were still relevant at the end of 
Spring, but Chloe's response for item 8 was very general and did not seem to be relevant to the concepts. 

Finally, statements 5, 10, 13 targeted the principle of motivation and relevancy of the information. 
Tammy's responses to all three items were very relevant for both semesters. Given Tammy's sensitivity to the 
motivation issue and her strong beliefs about the importance of motivation in learning, her “ very relevant” response 
to these items was predictable. Chloe's responses were also relevant but were not as elaborate and specific as 
Tammy's responses were. Given the fact that motivation is one of the major concerns for a preservice teacher, the 
participants' relevant responses may not have been due solely to instructional design training that they received. 

How do preservice teachers who are trained in ISD plan instruction in their education courses 
and during their student teaching? 

In order to answer the above questions we analyzed the lesson plans that Tammy and Chloe developed for 
their instructional design course, their tutoring tasks, their micro-teaching tasks and their student teaching. 

The first lesson plan that both Tammy and Chloe developed was in their ID course in which they used 
Gagne and Briggs' (1979) nine events of instruction as the framework for creating their lessons. Lesson plans began 
with a five component objective, a list of proper conditions for learning and then prescriptions for each event 
followed by a brief rationale. The objective, instructional strategies or conditions for learning and assessment 
strategies were aligned. Both students received a high mark for their first lesson plan. Since the lesson plan was part 
of a required assignment in the instructional design course (the first course in the two- semester-course sequence), 
both participants followed instructional design procedures and principles. 
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The next set of lesson plans was developed during the time that Tammy and Chloe were assigned to tutor a 
child. Chloe’ s lesson plans for her tutoring sessions were developed during Spring 1995 (the same semester in which 
she was taking some of her methods courses). Tammy completed her lesson plans during Fall 1995; and she, too, 
was taking methods courses. The lesson plan for each tutoring session was developed on a predesigned form which 
was slightly different for different subject matter (e.g., math and literacy). For example, in the literacy form, each 
tutor must identify the “observed strengths” of the tutee, goals of the session (lesson), activities and resource 
materials for observations. At the bottom of the form each tutor also completed the section labeled “reflections and 
connections.” The predesigned form for a mathematics lesson plan is more detailed and asks for: objectives (from NC 
standard course of study), teaching strategies, student strategies, session evaluation, implications for the next lesson, 
lesson resources, manipulates and student reaction. 

Analysis of both Chloe's and Tammy’s lesson plans for tutoring sessions showed that they wrote 
observable objectives, identified the tutee’ s prerequisite knowledge, and planned an activity for the stated objective. In 
the literacy lesson plan (since no place on the form was devoted to assessment (testing strategies), they did not 
indicate how they would assess their student’s learning outcomes. However, some of both Chloe's and Tammy’s 
reflection notes point to the strategies that each had used to assess the tutee’ s learning (e.g., “Clair was able to 
remember the pieces of the story”; “Clair read a story today. She was able to do it by herself. She only got hung up 
on a few words.” ). As indicated above, mathematics lesson plans were more detailed. In addition to objectives and 
teaching strategies, evaluation of each session was to be recorded (although not adequately explained in the example 
below). As noted on the lesson form, the state standards were to be recorded as the lesson objectives. Therefore, 
neither participant generated her own objective for their math lesson. Rather they adapted objectives from the list of 
state standards. The following is an example of such a mathematics lesson plan. 

Objective(s): The child will recognize patterns by playing tic-tac-toe and checkers (from NC state standards). 
Teaching strategies: I will explain one game at a time to Clair and then as we play I will point out strategies for 
her to use allowing her to win so that she can see the patterns and learn what to look for. 

Students' activities: Student will play tic-tac-toe and checkers in order to develop pattern. 

Session evaluation: The child did not comprehend how to play tic-tac-toe. She did do well at checkers and could 
understand . . .forward pattern movement. 

Implications for next lesson: I want to try tic-tac-toe again, but blocking off the board as follows. . 

The third set of lesson plans was developed during the student teaching internship. Both teachers used a 
lesson planning format known as the North Carolina Six Point Lesson Plan. The elements of this lesson planning 
guide are derived from Madelyn Hunter's direct instruction model for planning and teaching (1982) and include the 
following steps: 1) Focus and Review, 2) Statement of Objective, 3) Teacher Input, 4) Guided Practice, 5) 
Independent Practice and 6) Closure. Students learned this lesson planning model in methods courses after first 
acquiring some facility with Gagne and Briggs' (1979) nine events of instruction. All student teachers are required to 
prepare and submit their lesson plans to their cooperating teacher for review before execution of their lessons. 
University supervisors review current and past lesson plans during their observation visits. Analysis of Chloe's and 
Tammy's lesson plans during their student teaching reveals that they did not always use their own objectives for their 
lessons. Interview data with each participant and the university supervisors also confirmed that the state standards and 
the teacher handbook were the main sources of objectives, although teaching strategies for lessons delivered during 
student teaching were largely the unique creation of each participant. 

In Tammy's case, the school system in which she student taught had adopted a highly structured set of 
programmed materials for elementary mathematics. Therefore, she had to follow the prespecified objectives, teaching 
strategies, resource materials, tests and quizzes when developing lesson plans. Tammy had very little freedom to 
incorporate her own ideas or strategies into her lesson plans. Moreover, Tammy's lesson plans were expected to 
conform to those of her cooperating teacher, which in turn were mainly reproduced from the teacher handbook. Of 
course, there were a few lessons in which she generated her own objectives by integrating them with the prespecified 
objectives of the textbook or teacher's handbook. In her responses to questions about one of these independently 
created lessons, she indicated that she had thought about the objectives of the lesson and designed her teaching 
strategies by concentrating on students' interests and involvement in the activities. However, interview data with 
Tammy's supervisor also revealed that the lesson plans that Tammy developed integrating her own goals and 
objectives into the state goals tended to misjudge her students' current level of understanding. According to her 



supervisor, "She had a hard time designing a lesson at the students’ level. She either designed too easy or too complex 
activities for her lessons." 

Analysis of Chloe's lesson plans reveals lesson objectives that are stated with little precision, as revealed by 
this supervisor's comment, "It would be good to have more precision in your objectives and to make a distinction 
between the learning activity and the intended objective of your lesson." The performance or behavioral indicator of 
the intended learning is seldom specified; and not coincidentally, the degree of congruence between instructional 
activities and objective(s) is often overly general. There is some indication that Chloe equates participation with 
learning. Her overriding goal is to have students involved and through involvment, motivated. While such a goal is 
certainly defensible and more than that, desirable, it sometimes appeared that the relationship between intended 
learning and the learning activity was of secondary importance. 

Nonetheless, this student demonstrated considerable teaching competence as reflected by her very high 
evaluations that consistently praised the quality of student engagement, her enthusiastic delivery, skillfully led 
discussions, effective questioning practices, good lesson pacing and her use of meaningful reflective comments. This 
preservice teacher quite adeptly achieved her stated goal(s) as a teacher as expressed in her educational philosophy 
paper. In Chloe's words, 

"As a teacher, it is important for me to individualize instruction and make the learning experience a meaningful 
process to all students." " It is also important for me to properly assess my students’ abilities." " For my 
students to grow intellectually, they need to feel comfortable with their surroundings and take risks." "My job 
is to find activities which will motivate students to pay attention, stay involved and work together. The whole 
language philosophy allows the children to learn in a meaningful way and not become bored with worksheets. 
In my classroom, students will be given ownership of their work, explore their surroundings and know they are 
valued as indivduals. This is my goal as teacher." 

Discussion 

Both Chloe and Tammy came to the teacher education program with a set of beliefs about learners, learning 
and teaching. While it seemed that Tammy's and Chloe's beliefs did not change dramatically over the course of their 
two-year teacher education program, they refined, expanded and validated their beliefs by gaining theoretical and 
practical knowledge. Both applied certain concepts learned in the instructional design sequence courses, but they 
tended to emphasize different concepts and principles, ones that were more consistent with their entry beliefs. For 
example, Chloe seemed to develop a stronger belief in pre-planning and structure for her classroom. She used the 
concept of learner analysis and students' prior knowledge to improve her planning. Chloe also seemed to think more 
about how to teach and how to put theories into practice. In her writing assignment, Chloe tended to discuss 
methodological issues at the practical rather than theoretical level. This more analytical approach helped her develop 
strategies for her future practice. 

Tammy, on the other hand, focused on such concepts as "who is a learner," "what is the learner’s role in the 
learning process" and "how a learner learns". She seemed to develop a stronger belief in the concept of "otherness" 
and the importance of each learner's motivation in learning. And in contrast to Chloe, Tammy’s thinking seemed to 
remain more at the theoretical level, failing to make a transition to developing specific strategies for her own 
teaching practice. Thus for Tammy, although most of her written assignments and tasks were theoretically sound, it 
was not clear whether or not she had thought about how she would actually implement her ideas and plans for 
teaching. 

The results of the instructional design survey indicated that both Tammy and Chloe used instructional 
design principles, and to some extent procedures, thereby reflecting a systems perspective on instruction. Tammy's 
and Chloe's responses to the survey statements were very similar to those of instructional designers in Driscoll, et 
al.’s study (1994). A small decline in consistency of responses with principles of instructional design was detected 
for both participants at the end of Spring semester. Both participants also tended to give wordier, and in some cases, 
more elaborate responses; but the content of their responses was less related to an ISD perspective. Chloe’s 
explanations showed more decline in her application of instructional design principles and concepts than did 
Tammy’s. One possible explanation is that the level of exposure to the instructional design language decreased as 
each participant progressed to her methods courses. Since Chloe was taking more methods courses during Spring 
(Tammy was still registered in core courses), this decline would be more noticeable in her responses at the end of 
Spring. A related and more substantive factor is the absence of continued practice in a systems approach to planning 
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instruction in their methods courses. Still another reason might be students’ increased concern and appropriately 
intensive focus in methods courses on exactly how to teach various subject matter disciplines. 

Obviously, Tammy and Chloe learned from the instructional design sequence courses, but the planning and 
teaching schemata that they constructed are not the same. The concepts, knowledge, planning procedures and 
teaching strategies that they most relied upon do in fact differ. Observations of Tammy’s and Chloe’s practice 
teaching and review of their lesson plans reveal differences in their planning and teaching behavior. 

Analysis of Chloe’s and Tammy's lesson plans and their execution showed that the lesson format taught by 
the university, the cooperating teacher's style, and the textbook materials highly influenced the way both participants 
designed their lessons. It seemed that by following the specified format prescribed by the state and university, both 
Chloe and Tammy developed a routine in their thinking about designing instruction. Several components of 
instructional design were still part of their thought process (although not always written) when they designed their 
lessons and executed them. These components were the objectives of the lesson, instructional strategies to enable 
students to reach the lesson objective, and motivational strategies. They gave little attention to precisely specifying 
their objectives, nor was there evidence of an interest in achieving a clear alignment of type of learning with the 
objective of the lesson or congruence among objective(s), lesson strategy and assessment strategy. 

Furthermore, the lesson plans and the statements that these preservice teachers make as they proceed 
through the teacher preparation program reveal an increasingly complex but also integrated conception of the 
teaching-learning process. There is a heightened sensitivity to the environmental demands of teaching, specifically 
time and resource constraints, the physical characteristics of classrooms and the array of problematic student 
behaviors, academic, social and emotional. Coupled with deeper appreciation of these realities is an emerging ability 
to integrate the multifaceted elements of teaching that reflects the necessity for teachers to plan for the varied 
challenges of teaching captive but often reluctant learners. 

For example, in Chole's third interview she says, ”1 think that I really have to focus on how to teach so 
that everybody is on the same level, where some students are not going to be left behind.” ”... I have to make some 
adjustments in my teaching methods to reach them.” So first of all what I did was, I thought of a certain activity 
that would grab their attention because I knew that what my lesson plan was going to be on was to teach them 2- 
digit addition. How was I supposed to make that fun? How was I supposed to get them active in learning, ... that’s 
why I started with [money] ... for the motivation, and then I ... broke it up as far as how I was going to get them to 
stay involved with the lesson.” 

And in response to the reality of special needs youngsters Chloe comments, ”... now that they’re 
mainstreaming they’re bringing in all of these students that would have never been in the regular classroom.” 
"...I'm going to be held accountable for their education ... so I have to make different adaptations in my teaching in 
order to teach them.” ”... the students are very important; the lesson is also important but I have to make it right 
for the students so they can learn.” [Chloe did her student teaching in a full inclusion classroom where 25% of her 
students were officially classified as having a special need.] 

The model for planning that emerges is one that recognizes the importance of the ecology of the classroom 
and thus requires that in the design of instruction teachers give considerable attention to arranging instruction in 
accordance with the physical and social dynamics of classrooms and the characteristics of learners in groups. It also 
incorporates the reality of students’ widely divergent social, cultural, attitudinal and learning characteristics. Both 
teachers are drawn to accentuate the differences among learners. In their minds, the individual learner’s needs are of 
paramount importance. Consequently, their planning mirrors this attention to the unique needs of individual 
learners. 

In view of the patterns of behavior and thinking displayed by teacher interns in this study, it seems 
reasonable to suggest that applications of instructional design for teachers be carefully modified to achieve models 
that better account for the ecology of the classroom environment. It is essential that teachers have a robust 
framework for planning that meaningfully incorporates present day concerns for the widely diverse characteristics of 
students. Today's public school classrooms are comprised of youngsters who challenge teachers enormously. 
Instructional design models for teachers must address the significant motivational and learning challenges that are 
common to K-12 students. 

ID models must also acknowledge the natural tendency of teachers to move quickly in their mental planning 
to selecting activities that will grab their students’ attention, imagination and interest. Finally, it is important that 
ID models more effectively incorporate principles of constructivist views of the teaching-learning process if they are 
to achieve the promise of having significant utility for improving the planning, teaching and evaluation roles of 
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teachers. More flexible models for planning instruction as proposed by Willis (1995) may in fact better capture the 
process that many, if not most teachers come to use as practitioners. 
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Table 1. Guiding Codes Based on Systematic Model of Instructional Design 



Guiding Codes Using Instructional Design Principles and Procedures 

• Thinks about the learners' characteristics and the learning environment before attempting to do 
planning of instruction. 

• Determines content to be covered simultaneously or before writing goals and objectives 

• Writes clear objectives that incorporate learner needs and expectations 

• Determines assessment strategies before thinking of teaching strategies 

• Divides a complex task into smaller, achievable learning units and subunits in terms of primacy of 
events 

• Decides on teaching strategies after considering objectives and conditions of learning 

• Plans an environment that is conducive to learning objective 

• Plans instructional strategies that motivate the learner (gain attention, relevancy of the task, 
immediate feedback, building confidence) 

• Plans an assessment strategy that is congruent with the learning objective 



Table 2. Guiding Codes Based on Recur sivee Instructional Design Models 
1 Guiding Codes Using Recursive Instructional Design Principles 




392-413. 



• Focuses on the learner and learner's needs throughout the design, development and implementation 

• Focuses on outcomes of instruction (objectives/goals) 

• Determines content (what topics to cover) 

• Focuses on task to be taught and sequence of the task to be taught and the materials, facilities to be 
used 

• Elaborates on instructional description/strategies using Gagne's condition of learning 

• Demonstrates congruence between goals/objectives and instructional description/strategies and 
assessment methods 

• Demonstrates evaluation of instruction as well as students' learning outcomes both during and after 
instruction 

• There is a consistency between beliefs about learning and teaching, and the instructional strategies 
selected 



Table 3. Chloe's Beliefs and Dispositions at Different Points in the Teacher Education Program 



At beginning of Fall 
1994 


Fall 1994 


Spring 1995 


Before Student Teaching 


"... good teaching is 
when students are doing 
activities. . . ." 

". . .a good teacher is the 
one who makes things 
fun for students. . . 

"... teaching is hard but 
rewarding. . . I always 
reflect back to my years 
in school. It was never 
motivating. For example 
I never wanted to learn 
geography. I want to 
make things rich and 
exciting for students 
because it wasn't like that 
for me. . . ." 

"... I see myself as a 
mentor, counselor and 
friend, teaching my 
students about their 
capabilities. This is the 
goal that I hope to 
accomplish." 


". . . lesson plans should 
be completed before the 
first day of the class. . . ." 
"... I will tell my 
students the first day 
about classroom rules and 
the consequences of 
breaking them. . 

". . .students need to be 
taught from the beginning 
what is expected of them. 

. . I would assign easy 
tasks and activities at the 
beginning of the school. . 

. I will be specific in 
telling students what is 
acceptable and what is 
not. . . " 

"... while preparing my 
lessons, I must keep two 
important considerations: 
(1) the skills and concepts 
they must learn, and (2) 
through which activities 
they can best learn them. 

"... I must provide 
lessons that are broken 
down into steps, provide 
practice and encourage 
them. . . " 


"... I picked the 
metaphor of "keeper of 
the garden" for my 
teaching philosophy, 
because it is important 
to guide my students 
like a gardner who tends 
to flowers and takes out 
the weeds. . . 

"... I need to know 
about students’ 
prerequisite knowledge. I 
need this information to 
know on what level to 
begin. I also need to 
learn about students' 
interests. . . I gather 
information about 
students during 
planning. . . I use 
information collected to 
plan activities. . . I use 
information about 
students' interests to 
monitor students’ 
involvement. . ." 


. . this is an excellent 
strategy to use with young 
children because they cannot 
retain a large amount of 
information at one time. By 
breaking down the story 
into parts, the students were 
able to focus on smaller 
pieces of information. . . " 

". . .1 believe my job is to 
supply the needs of my 
students. . . classroom 
management is also critical 
in students’ learning. I need 
to post rules and 
consequences. . . I also 
believe in cooperative work 
with parents. . . ." 

"When I am in my student 
teaching classroom, I really 
want to implement fun 
writing activities for 
students, maybe getting 
more books on Native 
Americans. . . I really want 
to try portfolios with my 
kids. . . " 

I think behavioral 
management is the area that 
I might want to provide 
help to my cooperative 
teacher in order to 
implement what I know. . . 
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Table 4. Tammy's Beliefs and Dispositions at Different Poins During Teacher Education Program 



At beginning of Fall 1994 


Fall 1994 


Spring 1995 


Before Student Teaching 


. .Teaching is making 
someone else's life a little 
fuller by providing them 
with the know-how to be 
successful. . . I don’t want 
to expect too little but at 
the same time I don’t want 
to expect too much of my 
students because it is easy 
for kids to get frustrated. . . 

I see myself as a teacher 
who would get down to the 
floor with kids either in 
groups or as individuals. . . 

. I would like to look at my 
students from their level so 
as to understand them 
better, but I would also see 
them as intensively smart. . 

tt 

“. . .My fifth grade teacher 
was very positive. She 
never yelled at you, she 
always helped you out . . . 
She wasn’t so serious, she 
wouldn’t mind jumping 
around or just having fun . . 
. At the middle grade when I 
got a good grade, they 
would tell me that I got it 
because of my father. . . 
when I was in elementary 
school, I had some friends 
who were LD or else they 
were mentally retarded and I 
remember that I was their 
only friends. Nobody else 
would be friends with them 
... I used to be teased by 
other kids because of being 
friends with special kids. . 

. I was a tomboy and used 
to be teased a lot because of 
that, so I became really shy 
and quiet and either played 
with my brother and his 
friend or with guys at 
school 


“. . . I wasn’t raised in the 
states, but a lot of students 
agree that it’s okay to have 
corporal punishment in the 
classroom. I don’t think it 
is.. There are a lot of 
contradictions in 200 for 
me. . . I learned a lot from 
behavioral management. It 
showed me that I don’t 
want to be a behavior 
management teacher. . . 
Ms... is a good teacher, 
she really thinks about her 
students and asks herself 
the question “How will 
this affect them?”. . . 

When choosing activities 
for my classroom, I will 
choose those that I feel are 
relevant to learning and 
encourage the type of 
learning outcome I am 
looking for. . . I will do a 
lot of demonstrating, 
questioning and talking 
students through problems. 
. . . My greatest strength 
would be sympathy toward 
others, an appreciation of 
their differences, an ability 
to imagine their 
“otherness”. . We think we 
learn best when we are in 
the position to make sense 
of things. . . 

“. . . I would arrange 
activities that would build 
logically upon each other 
providing prerequisite for 
future activities. . . I 
would give students 
strategies that they can use 
individually to help them 
make sense of the 
materials. . . 


“. . . A good teacher is 
the one who is humble, 
a role model, reflective 
and uses techniques and 
principles of attention 
and making materials 
relevant and meaningful. 

. . I want to be known 
for motivating students 
and helping them to 
achieve. . . I feel that 
teaching humanity and 
respect for themselves 
and others is of equal 
value. . . Without 
attention, teachers will 
not teach students 
anything. . . I would use 
hands-on and explorative 
learning strategies. . . 


“. . . There is much 
more to teaching than 
textbook. . . Learning 
needs to be explorative 
in nature as well as fun 
and meaningful. . . 
Learning is surfing life. 
It means learning 
includes all types of 
learning and subjects yet 
is specific in nature. . . 





Table 5. Tammy *s Responses to the ISD Survey 



Survey Items 


End of Fall 1994 


End of Spring 1995 


l.'T can't believe we're starting 
something different when I don't 
even understand what we just did. 

( target response : Remediation and/ or 
prerequisites) 


In this case the teacher could have 
asked if there were any questions as 
she/he moved through the lesson so 
as to clear up anything that wasn’t 
understood. She also could have 
given them some kind of work that 
she would need to have checked for 
their understanding. 


After the teacher had completed the 
lesson she/he should have asked if 
anyone needed further explanation 
and either provided it on a one to one 
basis or in the whole class setting 


2. "I turned in my answers to the 
homework problems he gave us, but 
I never found out how I did." 

(Target response: Feedback for the 
practice ) 


This teacher needs to provide 
feedback. He could do this on an 
individual basis or correct the papers 
and hand them back. 


The teacher needs to return the 
homework so that students get 
feedback. They also need to be told 
how they did so they can judge their 
own understanding of the topic. The 
teacher could have made their 
homework available to them by 
handing it back or letting them know 
they could look at it. 


3. “I guess I blew that one. More 
than half the questions were on a 
topic the teacher barely touched on in 
class.” 

( Target response: Alignment between 
test items/assessment strategies and 
objectives) 


If so much of the test was going to 
be based on topics not covered in 
class, then the teacher should have 
let the class know that they needed to 
really study that and ask any 
questions they had about it, or she 
could have just had one or two 
questions on that topic. 


When making the test out the teacher 
needs to refer back to her lesson 
plans and make sure she covered all 
the material that she wants to ask 
questions about. She could also let 
the student know in advance if the 
test is going to come from class 
discussion or textbook readings. 


4. “Even after looking at my notes, 
I still couldn’t do the homework 
problems.” 

( Target response: Example and/or 
Explanation or demonstration) 


The teacher probably needs to be 
more organized and more explicit. In 
her lessons she should have provided 
many examples and non-examples 
for the students to refer to. 


The teacher needs to make sure she 
has completely explained the lesson 
in an orderly fashion, not straying 
off the main point so as to confuse 
the students. She also needs to make 
sure the students understand the 
lesson before they go do homework. 


5. “I really liked this last unit. I 
wish we could do more on it.” 
(Target response: Motivation and/or 
Students * interest) 


The teacher needs to provide for 
further learning. Maybe give some 
enhancing retention assignments or 
give the student some extra work 
(depending on age). 


Have a learning center available for 
the students, and the teacher needs to 
let students know when and how 
they can do some self exploration if 
the subject interests them. 


6. "Boy, those questions were 
awfully tricky. They really didn't get 
at what I know." 

(Target response: Conditions of the 
objective , testing) 


The teacher needs to be more precise, 
not trying to trick the student. 

He/she also needs to present the 
material and provide practice in a 
way as not to confuse or trick 
students when it comes to the test. 
The test should be similar to in-class 
activities. 


The teacher needs to focus on what 
has been taught, not on trying to 
trick the students. Her questions 
need to be direct and well stated. To 
avoid this she could have one or two 
people pre-read her test before she 
gave it to the class, or use it as a 
sample test the previous year. 


7. “I wonder what we’re supposed to 
be learning from this chapter.” 
(Target response: objectives) 


At the beginning of the lesson the 
teacher needs to inform the learner of 
what is to be covered in this chapter 
and what the student will be able to 
do and know when they are finished. 


At the beginning of the unit the 
teacher could state what his/her 
objectives are and explain briefly 
what the chapter is about and why it 
is relevant information. 





Table 5 continues. Tammy's Responses to the ISD Survey 



Survey Items 


End of Fall 1994 


End of Spring 1995 


8. “She asked us to do those 
problems but never told us how to 
do them.” 

(Target response: Information and/or 
Examples) 


The teacher needs to show students 
how to work problems, providing 
examples and nonexamples, 
answering questions and maybe 
working through one of the 
problems before asking students to 
do them on their own. 


If she is testing for prior knowledge 
then she needs to let her class know 
by stating that, “this is a pre-test, 
just do what you can.” But if she is 
giving this as an assignment then 
she needs to teach the material 
beforehand and have the students 
practice the kind of problem before 
they go to do it on their own. 


9. “I don’t know how I’m supposed 
to understand this. Everybody else 
seems to know something that I 
don’t.” 


Eliminated 


Eliminated 


10. “I have no idea why we are 
studying this topic. It doesn’t seem 
to have anything to do with what 
I’m interested in.” 

(Target response: motivation) 


The teacher needs to inform the 
learner of the relevance of the 
information, making the student 
aware of what is being covered, how 
it will affect them and why. 


The teacher needs to explain the 
relevance the topic has in the future 
for her students. She/he also needs 
to make the material meaningful by 
providing examples and non- 
examples to the students. 


11. “All she ever does is lecture. 

We never get a chance to do anything 
before we’re tested on it.” 

(Target response: Practice and 
feedback) 


The teacher needs to give the 
students a chance to practice what 
they are learning and then provide 
feedback on how they are doing. She 
also needs to ask the students if they 
have any questions or things they 
don’t understand. 


The teacher needs to provide time for 
practice! She also needs to make 
sure that the students understand 
what is being practiced by having 
class discussions and some 
assignments to do. She cannot 
expect the students to do well if they 
have never practiced what is being 
taught. 


12. “He talks and he talks and he 
talks, and we have no idea what 
we’re supposed to be learning!” 
(Target response: Objective) 


This teacher needs to be more 
explicit, stay on task and inform the 
students as to what they re going to 
be learning and what the outcome of 
the lesson will be. 


He needs to focus on his topic and 
not get off on a tangent. He could 
make an outline or list of points for 
himself to follow. He also needs to 
check with his students for 
understanding by asking them 
questions and vise versa. 


13. “This is so boring. I just don’t 
like this stuff!” 


Eliminated 


Eliminated 


14. “I wish the teacher would spend 
more time with me so I could 
understand this stuff before we move 
on.” 

(Target response: Individualized 
instruction) 


This teacher either needs to cover the 
material more clearly, making sure 
that students are left without 
misunderstandings, or she/he needs 
to make themselves available to the 
student for extra help. 


The teacher needs to make 
herself/himself available to those 
students who need extra help. 
She/he could do this by having 
weekly 5 minute conferences with 
each student or using a portfolio in 
order for students to address their 
needs then make sure she/he helps 
each one either individually or as a 
small group before they move on. 
By doing this she will provide a 
firmer background for them. 




230 






o n 

cb 



Table 6. Chloe 's Responses to the ISD Survey 



Survey Items 


End of Fall 1994 


End of Spring 1995 


l."I can't believe we're starting 
something different when I don’t 
even understand what we just did. 
(target response: Remediation and/or 
prerequisites) 


The teacher didn’t introduce the 
objective and why it was important. 
If he/she would have done this and 
also tied the new objective in, the 
transition may have been easier for 
this student. 


Basically, this teacher did not listen 
to her students. This reminds me of 
a teacher who is more interested in 
covering so much material, that she 
is not interested in student 
understanding. 


2. ”1 turned in my answers to the 
homework problems he gave us, but 
I never found out how I did." 

(Target response: Feedback for the 
practice) 


This teacher lacked to give students 
immediate feedback, which is crucial 
to student learning. As teachers, we 
expect homework to be turned in by 
a certain date and we should also be 
expected to grade papers and return 
them as soon as possible. 


This response is coming from a 
student who wants to know how 
well they performed on their 
homework problems, and is now 
frustrated that they haven’t been 
returned. This has major 
implications on learning because it’s 
very difficult to “unlearn” something 
that is being repeated by students. 


3. “I guess I blew that one. More 
than half the questions were on a 
topic the teacher barely touched on in 
class.” 

(Target response : Alignment between 
test items/assessment strategies and 
objectives) 


The teacher failed to test what she/he 
taught. Tests aren’t given to trick 
students, but to reveal what has been 
taught to students. 


. This teacher needed to take a course 
in EDN 302! Tests are not meant to 
trick students but to assess what they 
learned in class. The test should also 
be set up so the students perform the 
material the same way they learned it 
in class. Example: In class students 
demonstrate how to solve a problem. 
The test should also have students 
demonstrate how to solve the 
problem. 


4. “Even after looking at my notes, 
I still couldn’t do the homework 
problems.” 

(Target response: Example and/or 
Explanation or demonstration) 


This teacher should have given 
problems that were as difficult as 
those done in class. Homework 
problems are given so they practice 
what they’ve learned. 


The teacher needs to develop several 
strategies on how to teach this 
subject. It’s obvious that this 
student still cannot solve the 
homework problems with the lecture 
from the teacher. Changes in 
instruction need to be made. 


5. “I really liked this last unit. I 
wish we could do more on it.” 
(Target response: Motivation and /or 
Students ’ interest) 


This teacher probably made this 
particular unit fun and interesting. 
This should be a cue that students 
want to be motivated to learn. 


This is a perfect opportunity for a 
teacher! She can find out this 
student’s interests and motivate this 
student to learn more about this unit. 
She can support this student’s 
learning by supplying a variety of 
books about the particular unit. 


6. "Boy, those questions were 
awfully tricky. They really didn’t get 
at what I know.” 

(Target response: Conditions of the 
objective ; testing) 


This teacher tried to trick students 
with the types of questions he/she 
gave or the wording was ambiguous 


This teacher did not make a test that 
correlated with the concepts taught in 
class. As a teacher, I need to 
emphasize certain points that I feel 
are important, so students won’t 
react to a test the same way this 
student did. This teacher also failed 
to review those concepts that would 
be found on the test. Tests are not 
meant to be tricky. 
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Table 6 continues. Chloe's Responses to the ISD Survey 



Survey Items 


End of Fall 1994 


End of Spring 1995 


7. “I wonder what we’re supposed to 
be learning from this chapter.” 

( Target response: objectives) 


The teacher didn’t properly introduce 
the lesson and why they were 
expected to learn it. 


This teacher failed to make an 
introductory statement that summed 
up all the concepts of the chapter and 
why they were important. Children 
need to know why they are doing 
certain things, or problems could 
occur. 


8. “She asked us to do those 
problems but never told us how to 
do them.” 

( Target response: Information and/or 
Examples) 


This teacher failed to thoroughly 
explain and demonstrate in class how 
to solve the problems. She needs to 
spend more time giving examples 
and non-examples 


This teacher failed to teach children 
how to solve these problems. This 
student will feel less motivated to 
learn and probably feels frustrated 
because of what this teacher did. 
This also shows there is a lack of 
communication between the teacher 
and her students. 


9. “I don’t know how I’m supposed 
to understand this. Everybody else 
seems to know something that I 
don’t.” 


Eliminated 


Eliminated 


10. “I have no idea why we are 
studying this topic. It doesn’t seem 
to have anything to do with what 
I’m interested in.” 

( Target response: motivation) 


The teacher should mention why 
they were learning this topic and also 
tell the student that not everything in 
school may appeal to all students. 


This teacher failed to acknowledge 
her students’ interests, which is a 
great motivator. The teacher could 
have made an interest sheet or asked 
her students about those subjects 
they really enjoy learning about. 
This works as a great thematic unit. 


11. “All she ever does is lecture. 

We never get a chance to do anything 
before we’re tested on it.” 

(Target response: Practice and 
feedback) 


Unfortunately, this teacher is having 
all the attention focused on her. She 
needs to allow these students to have 
fun exploring different topics using 
manipulatives. 


This is a traditional teacher who is 
basically talking throughout the 
whole day. Unfortunately, her 
students are bored and don’t want to 
listen anymore. She needs to get her 
students more active and involved in 
their learning. Once this occurs, she 
will be amazed at the results. Her 
students 


12. “He talks and he talks and he 
talks, and we have no idea what 
we’re supposed to be learning!” 
(Target response: Objective) 


This teacher is just rambling and 
doesn’t have clear objectives for his 
students. Students need to be told 
what they are learning and why. 


This teacher is not organized as a 
teacher. He needs to focus his lesson 
on certain topics and involve the 
student with the types of questions 
he asks. 


13. “This is so boring. I just don’t 
like this stuff!” 


Eliminated 


Eliminated 


14. “I wish the teacher would spend 
more time with me so I could 
understand this stuff before we move 
on.” 

(Target response: Individualized 
instruction ) 


This teacher hasn’t noticed the needs 
of her students. This could be done 
while checking papers and also 
talking with the student. 


Teachers need to set aside a specified 
amount of time where she can 
interact individually with her 
students. The discussion could cover 
a wide variety of subjects or projects 
they might be currently working on. 
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Introduction 

Without doubt computer technology can enhance education, but it is most powerful when it is truly 
integrated into the curriculum. Effective use of computers in the classroom requires curriculum transformation in the 
schools, which in turn requires technically and instructionally well-prepared teachers. Reports made on computer 
usage state that increased availability of computers does not always lead to improved utilization. In spite of the 
increasing number of computers in schools (Ely, 1989; U.S. Congress, 1995), not many practicing teachers feel that 
they have adequate training to enable them to integrate computer technology into their curriculum and instruction 
(Faison, 1996). Recent research on the role of teachers in adoptation of computers into their instruction highlights 
several factors that influence teachers’ usage of computers. These factors include: (1) the adequacy of teachers' training 
in computer use, (2) the level of administrative support they are likely to receive, (3) the presence or absence of 
curriculum constraints, and (4) teachers' personal preferences (Beaver, 1990; Cuban, 1989; Stover, 1990; Wiske, et 
al., 1990). Of all these factors, adequate teacher training in computer use (Gressand & Loyd, 1985; Phillips, 
Nachtigal, & Hobbs, 1986; Thompson, 1985; Wilkinson, 1980) remains to be the most influential and among the 
highest needs of schools (Stiegemeier, 1995). 

Another body of literature on school staff development training in microcomputers indicates that training 
workshops or formal coursework in computers do not solely affect teachers use of computers (Sibalwa, 1983). There 
needs to be a careful and systematic plan for training teachers at the school and district levels with respect to 
integration of computers and related technologies in the curriculum (NASSP, 1994). Reports also point to the need 
for on-going, site-based technical and instructional support systems for teachers to ensure the continuous use of 
computers and related technology (e.g., Brennan, 1991; Fulton, 1988; Wisconsin, 1990). 

During the past few years, many school districts around the country have responded to the above needs by 
developing a district- wide plan for the integration of computers and related technologies into the curriculum. A part 
of this plan has been to establish a district and a school-based technical and instructional support system to assure 
the proper usage of computer technology for instructional purposes. This attempt has also resulted in the creation of 
a position that seems to expand the utilization of media and library resources (as used by media specialist), and 
requires skills and knowledge in instructional technology. This new human resource position is entitled Technology 
Coordinator (at the district level) or Technology Resource Teacher (“TRT’) at the school level, and is responsible for 
helping the district or the school integrate technology into classroom instruction. It is interesting that the public 
schools are now beginning to acknowledge the pool of educators who have been trained in instructional design and 
technology after decades of overlooking the need for their role. The result of this attempt by the public schools is 
extremely important in shedding light on the need for the instructional technologist in the schools, and to the future 
role of the instructional technologist in the public school. 

During Spring and Fall 1995, Moallem, Mory and Rizzo (1996) conducted a study to identify the roles and 
responsibilities of a group of TRTs at the middle school level in a school district in southeastern North Carolina. 
This study examined the extent to which the TRTs have been successful in helping classroom teachers use and 
integrate computers in their curriculum and instruction. The study investigated how technology resource teachers' 
roles differed from those expected of instructional designers/technologists. The results of this study suggested that 
although TRTs reported that they spent 75% of their time on technical support, the nature of the TRTs' role was 
more instructional than technical. In addition, the above study indicated that the TRTs’ long term goals were to 
assist teachers integrate technology into their classrooms and to train teachers to train students in computer usage. 

The result of the above study also demonstrated that TRTs were able to develop a positive and collegial 
relationship with teachers in order to help them integrate technology. However, it did not show whether the teachers' 
attitude, their usage of computers, or their classroom practice had changed as a result of TRTs support. This was 
especially important since the above study indicated that the TRTs did not appear to have the skills and knowledge 
that an instructional technologist would have if he/she were in this position. It should be noted that the TRT 



position was first instituted in September 1994, and the above research study was conducted at the end of the first 
year of the institution of the computer competency program and the TRT position. 

The Purpose of the Study 

The purpose of this study was to assess the progress and effectiveness of TRTs, as technical and 
instructional support persons, in their effort to assist classroom teachers to integrate computer technology into 
curriculum and instruction. It examined the roles and responsibilities of TRTs to identify any changes in their roles 
and approaches in achieving their long-terms goals. It also aimed to compare the new results with the previous 
results (Moallem, Mory and Rizzo, 1996) in order to identify the future roles of instructional technologists in 
public schools. 

The study specifically focused on the following questions: 

• How did the role of TRT change over time (from the first year to the second year)? 

• What problems did TRTs encounter in helping teachers integrate technology into their classroom? and What 

strategies did they use to alleviate the problems? 

• How were TRTs perceived by teachers? and What effect did the TRTs have on the teachers’ integration of 

computers into their curriculum and instruction? 

• How did the teachers feel about computer technology, and was there any change in their attitudes toward the 

computer and its usage in the second year? 

• What was the level of usage of computers by teachers, and how did it change over the past year? 

Methodology 

The study was conducted using an explanatory case study approach within a naturalistic paradigm utilizing 
both qualitative and quantitative methods of inquiry (Yin, 1994). We perceived this approach to be appropriate for 
the questions that we asked. We employed a multiple data collection technique to triangulate the information from 
different sources and to ensure accurate portrayal of reality. By using this approach, we were able to employ multiple 
data-gathering methods. Since we intended to compare the data of the first year with the data of the second year to 
determine the change, we collected data using similar methods and schedule at the end of first and the second years. 
The data-collection methods included: 

• informal and formal interviews with technology resource teachers, 

• classroom, computer lab and site observations, 

• public and personal records, reports and documents, and 

• teacher questionnaire and attitude surveys. 

Due to the limitation of time and resources and because the program was first conducted at the middle 
school level, as with the first year study, we focused on the middle schools in the county (six in total). Our data 
collection procedure began in early February and continued until early September, 1996. From February to March, 
1996 we interviewed the TRTs and observed the computer facilities and computer labs in each school. During the 
month of May, 1996 we observed computer labs while in use by classroom teachers. We also administered the same 
surveys which we used at the end of the first year. The General Information Survey (the “general survey”) was 
constructed by the researchers around the major questions of the study, and the Computer Literacy Attitude Survey 
(“CL A”) was adopted (Savenye, 1992; Savenye, Davidson, & Orr, 1992). Both survey instruments were completed 
by teachers following one of their respective school's professional development meetings. 187 teachers responded to 
the CL A survey (compared to 206 in the first year), and 217 teachers answered the general survey (compared to 176 
in the first year). The return rate for both the general survey and the CLA survey was between 90-95% for two of the 
schools, and was between 55-71% for the other four schools. 

The general survey consisted of 18 closed-ended and 9 open-ended items. The closed-ended items asked about 
teachers' computer facilities at home and the classroom, their usage of their computer facilities, their previous and 
present computer training, the computer competency tests, and their feelings about their computer skills. The open- 
ended items, on the other hand, asked teachers about their perception of the role and responsibilities of the TRT, and 
the type of help that teachers received from the TRT in their respective schools. The CLA survey consisted of 50 
Likert-scale type items which were developed and used by Savenye and her colleagues (Savenye, 1992; Savenye, 
Davidson, & Orr, 1992) to measure attitudes of preservice teachers towards computers. The items were related to 



liking computers, valuing computers for society and education, anxiety about using computers, confidence with 
regard to learning and using computers and perceptions of gender appropriateness of computers. Teachers were asked 
to rate the items from “Strongly Agree” to “Strongly Disagree”. In addition to Likert-scale items, teachers were also 
asked several background questions. The items were slightly modified to be used with in-service teachers. In 
addition to the Likert-scale items, teachers were asked several questions about their background and the number of 
hours that they had participated in computer training workshops. 

Data Analysis 

We analyzed the data using both qualitative and quantitative analysis techniques. Interview and observation 
data and the results of the open-ended questions from the general survey were analyzed qualitatively using the Miles 
and Huberman (1984) model. Based on this approach, the first part of the analysis was data reduction. During this 
process, the data chunks were identified and coded, the patterns that best summarized a number of chunks were sorted 
and then were further subsumed into larger patterns. In some cases, the data were organized using the frequency of 
the responses to specific questions or by the pattern of responses. In such cases, however, the numbers were used 
together with the words to keep the data in its context. During the second analytical stage, the data were summarized 
and organized using matrices, charts and tables. This stage helped the researchers interact with the data and draw their 
preliminary conclusions, which in turn triggered another round of testing and verification using different sources of 
the data for the final conclusion. 

The results of the closed-ended items of the general survey and the CLA survey were analyzed quantitatively. 
Descriptive statistics, cross tabulation, Chi-square, and Multivariate Analysis of Variance (“MANOVA”) were used 
for this part of the data. Since the number of teachers who responded to the two surveys were not equal across the 
schools, we randomly selected an equal number from each school to conduct cross tabulation, chi-square and 
MANOVA. 

Results 

Description of the School District and the Technology Initiatives 

The study was conducted in all of the middle schools (a total of six) within a city district in the southeastern 
United States. The district is within a zone that draws from affluent neighborhoods, as well as from neighborhoods 
of racially- and ethnically-mixed working- and low-class families. 

In 1992, the State of North Carolina Department of Public Instruction put together a set of computer skill 
competencies as a basic requirement for public school teachers and students. In anticipation of this state technology 
initiative requiring all schools to integrate technology into their classrooms and all teachers and students to pass a 
technology competency test, the district Technology Coordinators designed a plan to enable the teachers and students 
in the district to attain the necessary computer skills in accordance with the state technology initiative. The 
elementary schools in the district already had a designated classroom teacher to act as a technology resource person, 
and the elementary classroom teachers were already using technology in the classroom to some degree. The middle 
schools in the district, however, were not set up with such a situation. The mandate included a timeline to 
eventually administer a test to all eighth graders in the state to insure that the students had met the minimum 
computer skills. It was imperative, therefore, that the middle school teachers and students be targeted for broader 
technology support. 

The TRT position was created to provide technical and instructional support at the school level to help 
teachers integrate technology into their curriculum and classroom practices, and to assist teachers in passing the 
technology competency test required by the state. This position was created to be different from the media specialist 
position which already existed in most schools. The role of the TRT differs from that of the media specialist 
because the TRT was expected to be responsible for integration of computer technology into classrooms, while the 
media specialist was responsible for library media and providing print and media support at the school library. Two 
Technology Coordinators at the district level were in charge of developing a strategic plan and a job description for 
the TRT position and helping principals hire the qualified individuals. These two coordinators were also in charge of 
preparing the TRTs for their job responsibilities by providing them with training workshops and proper technical 
and administrative support. 
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Technology Resource Teachers (TRTs) 

Out of the six original TRTs, two left their positions at the end of the first year, and these positions were 
filled by two new individuals from within the respective school. The educational background of the TRTs spans from 
1-22 years of teaching experience. The majority of them have a background in teaching math (66%), science 
(16.6%), or special education (16.6%). Four of the TRTs were classroom teachers in their respective schools before 
being appointed to this position. The other two TRT were hired from outside the school. In general, the TRTs’ 
background in computer training incorporates college courses and in-service training workshops (50%), job related 
experiences (33.3%) and self-leaming/practices (16.7%). Each of the TRTs have passed the computer competency 
test required by the state of North Carolina. Their reasons for becoming TRTs include an interest in technology and 
teaching. None of the TRTs, however, have had any training or college courses in instructional design and 
technology. They also have not had any training and/or experience in analysis, design, development and/or evaluation 
of instructional materials and/or programs. 
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How did the TRTs' role change over time ? What problems did TRTs encounter in helping 
teachers integrate technology in their classroom and what strategies did they use to alleviate 
the problems? 

The observation, interview and questionnaire data showed that the nature of the TRTs’ role remained mainly 
the same as compared with the first year. All six TRTs continued to maintain the computer lab operation (technical 
support) and to provide training workshops for teachers and students (instructional support). However, the focus of 
the TRTs’ responsibilities changed by the end of the second year. Compared to the first year in which the TRTs 
spent more time on technical issues (75%), rather than instructional (25% ). By the end of the second year they 
reported spending more time on instructional matters. Moreover, the TRTs’ main emphasis on training teachers to 
pass computer competency tests shifted to preparing students, particularly eighth graders, to take the computer 
competency test in the second year. 

Analysis of other sources of the data provided explanation for this change. Since a large number of teachers 
were able to pass the computer competency test at the end of the first year, the TRTs were able to redirect their focus 
to provide training for students in the second year. In addition, since the number of teachers who were able to 
integrate technology into their classroom was still very low (between 2 to 5 individuals in each school) by the 
second year, the TRTs assumed the responsibility of preparing eighth graders for computer competency test in 
accordance with the state requirements. 

As with the first year, the main focus areas in the second year's computer workshops were: database, word 
processing and spreadsheets. Terms and operations, keyboarding, societal uses, and ethics were also taught. The 
percentage of teachers across six schools who received 30-40 hours of training workshops increased from the first 
year (28.4%)to the second year (37.8%) by 9.4%. Although no significant difference was found among the schools 
with respect to the number of training workshops teachers completed in the first year, there was significant difference 
among the schools in the second year (Chi-square = 6.7, p c.Ol). This result indicates that teachers in some schools 
completed more training workshops than in other schools in the second year. The interview data attests that the 
difference was due to several factors: (a) TRT’s plan of action, (b) principal’s emphasis on competency test, and (c) 
lower number of teachers who passed computer competency test. 

Analysis of data collected in the first year showed that in response to the question ”In what ways has the 
TRTs helped you with computer usage?” teachers listed the following items as the major areas of assistance: (1) 
find/update software, (2) available to help with what they need, (3) teach them how to use computer or pass 
computer competency test, (4) help with staff development and workshops, (5) troubleshoot, and (6) answer 
questions about technology. Analysis of data collected in the second year showed slightly different answers for this 
question. While teachers still listed the above areas of need, the majority of those who responded to this item 
(19.9%) indicated that TRTs helped them integrate technology into their teaching and lesson planning. Another 
difference was also related to TRTs’ assistance with computer competency test training. In the first year more 
teachers (15.2%) reported that the TRT helped them in computer competency test training than did teachers in the 
second year (7.4%). These differences demonstrate the change in the TRTs’ instructional role from the first year to 
the second year. As TRTs mentioned, since a greater number of teachers passed the computer competency test in the 
second year, TRTs did not see any need to provide more assistance in computer competency' training. Instead they 
were able to focus more on integration of computers in the classroom. 

The analysis of data in the first and second years showed that The TRTs’ primary instructional strategy for 
accomplishing their goals remained more or less the same. TRTs provided training workshops for teachers and tried 
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to model the teaching of the various computer competencies to students through an integrated lesson (e.g., language 
arts, mathematics, social studies, and science). The TRTs taught the majority of integrated lessons while the 
classroom teachers were present to help or to observe. In a few cases, the teachers themselves taught the integrated 
lessons with the TRT available to assist. 

At the end of both the first and second years TRTs were asked what problems they encountered in helping 
teachers to integrate technology. Their responses encompassed the following problems: (a) teachers' expectations of 
TRTs to assume responsibility for teaching technology to students, (b) teachers' resistance to learn and use 
computers, (c) teachers’ intimidation by computers, and (d) lack of proper hardware and software for teachers to use. 
Each TRT also reported using different strategies to solve the above problems. For example, some TRTs chose 
strategies such as: (a) providing more workshops (to increase teachers' computer skills), (b) giving more concrete 
examples and developing more meaningful activities for computer usage, (c) trying not "to step on teachers’ toes" 
and proceeding slowly, and (d) adding a sense of humor to make the process more comfortable. Some others took a 
more punitive approach by using strategies such as : (a) demonstrating to teachers the consequences of their 
resistance, (b) removing their classroom computers until they learned the skills, and (c) asking administrators to 
pressure teachers to take more responsibilities. 

How were TRTs perceived by teachers, and what effect did TRTs have on the teachers’ 
integration of computers into their curriculum and instruction? 

Comparison of teachers' perceptions of TRTs over the two-year period revealed that although there were 
some differences in the way teachers at each school perceived TRTs, there was more commonality than difference in 
their perceptions. For example, in the first year, the majority of teachers at each school described the TRT as 
someone who: (a) provides computer training or workshops for teachers and students, (b) assists in learning 
computer competencies or computer usage, (b) is always there to help them, and (c) is a problem-solver or a trouble 
shooter. In the second year, teachers still perceived the TRT as providing instructional support, helping teachers and 
students with learning computer competencies and as a helper and trouble shooter. 

TRTs were also asked to describe their perceptions of their roles and responsibilities. Comparison of the 
statements used by each TRT to describe his/her role over time revealed some differences. For example, at the end of 
the first year, the majority of TRTs (83%) believed that their role was more technical than instructional. They 
indicated further that they spent 70% or more of their time providing technical assistance and support (e.g., setting 
up and maintaining the lab, troubleshooting, installing software and hardware), and only 30% or less of their time 
providing instructional support. However, TRTs’ statements of their role in the second year indicated that they saw 
their role as being more instructional than technical. All of the TRTs indicated that their role was to provide 
workshops for teachers and students, help teachers integrate technology, teach in the computer lab and/or support 
teachers in their attempts to integrate technology. 

How did the teachers feel about computer technology, and was there any change in their 
attitudes toward the computer and its usage? 

A total of 206 teachers across the six middle schools completed the CLA survey at the end of the first year, 
and a total of 187 teachers completed CLA survey at the end of second year. The CLA survey contained items 
related to liking computers (r = .80), valuing computers for society and education ( r= .62) ; anxiety about using 
computers (r =.91); confidence with regard to learning (r = .67) and using computers (r = .73); and perceptions of 
gender appropriateness of computers (r = .47). Teachers were asked to rate the items from “Strongly Agree” to 
“Strongly Disagree”. In addition to Likert- scale items, teachers were also asked several background questions. 

Appendix A summarizes the means of all teachers' responses to the CLA survey over a two-year period. As 
mean scores show teachers' attitudes improved slightly in some areas. For example, teachers seemed to like 
computers better ( 24.36 to 25.57) and to have less anxiety about using computers ( 27.74 to 27.95) and more 
confidence about learning computers (3.38 to 3.42). However, teacher’s attitudes did not improve, and even declined 
in some other areas. For example, teachers did not seem to have more value for computers in education (55.39 to 
54.52), or the society (10.7 to 10.6) and did not change in their confidence about using computers (4.09 to 3.96). 
Teachers' perceptions of gender differences in computer usage also did not show any improvement (19.8 to 19.72). 

We examined differences in five measures of attitude across dependent variables, such as: different schools, 
the usage of computers in the classroom, different levels of previous computer experiences, and different levels of 
previous and present computer skills using MANOVA. Appendix B shows the means and standard deviation for 





each dependent variable and for each year. The result of the multivariate and univariate F values with significance at 
the .05 level or greater are also presented in Table 1 and 2 

As Table 1 and 2 show, there was no significant main effect in the variables, such as: liking computers, 
value for computers in education, confidence for learning and using computers, and anxiety for computers across 
different school in both years. 

We also tested differences in the usage of computers across the five measures of attitude. Appendix B shows 
means and standard deviations for each dependent variable in the different years. As Tables 1 and 2 show, the 
MANOVA yielded a significant main effect [Wilks’ Lambda = .9, F(5, 122) = 10.5, p <.05] for the first year and no 
significant main effect for the second year. The univariate analysis showed that teachers who were currently using 
computers in their classroom liked computers more in both the first year and the second year and had more value for 
computers in education in the first year. The anxiety and the confidence for using computers were not significantly 
different for those who used or did not use computers in their classrooms in the first year. However, the analysis of 
univariate for the second year showed teachers who were currently using computers had more confidence about using 
computers and had less anxiety about computers. 

Tables 1 and 2 show that MANOVA yielded a significant main effect for previous computer experience for 
both first year [Wilks' Lambda = .6, F ((5, 122) = 2.6, p <.001] and the second year [Wilks’ Lambda = .4, F (5, 108) 
= 4.1, p < .001]. The univariate analysis revealed that both in the first and second years, teachers who had previous 
computer experience liked computers more , had more confidence in learning computers, had more confidence in 
using computers, and showed less anxiety about computers. 

As shown in Tables 1 and 2, MANOVA yielded a significant main effect for rating computer skills before 
the school year in both the first [Wilks' Lambda = .5, F (5, 122) = 3.9, p < .001] and second years [Wilks’ Lambda 
= 5, F (5, 108) = 3.6, p < .001]. The univariate analysis indicated that in both the first and second years, teachers 
who rated their computer skills high before the beginning of the school year: liked computers more, had more 
confidence in learning and using computers and had less anxiety about computers. However, analysis showed no 
significant difference in value for computers in education for either years. 

Finally, as Tables 1 and 2 reveal, MANOVA yielded a significant main effect for rating present computer 
skills in both the first [Wilks' Lambda = .4, F (5, 122) = 5.3, p < .001] and second years [Wilks’ Lambda = .4, F (5, 
108) = 4.8, p < .001]. The univariate analysis showed that in both first and second years, teachers who had rated their 
present computer skills high liked computers more, had more confidence in learning and using computers and had 
less anxiety about computers. However, the univariate showed no significant difference in value for computers in 
education for either year. 

How did teachers feel about their knowledge and skills in computers? What was the level of 
usage of computers by teachers, and how did it change at the end of the second year? 

The general survey was completed by a total of 176 teachers in the first year and 217 teachers in second year 
respectively. The comparison of the results from the two years showed an increase in the number of teachers who 
have a home computer (51.5% to 58.5%) and are using their home computers for either school (50.6% to 55.9%) or 
personal purposes (51.7 to 53.0). The results also indicated an increase in the number of teachers who completed 
between 30-40 hours of training workshops (28.4% to 37.8%) and those who passed different areas of the computer 
competency test (see Table 3). Significant relationship was also observed between the hours of training workshops 
received and the teachers’ confidence about their computer skills (Chi-square = 16.4, p <.001). A high percentage of 
teachers who completed between 30-40 hours of computer workshops indicated that they felt confident about most of 
the computer skills. 

However, with respect to the classroom computers, although TRTs indicated that almost all teachers in 
their respective schools had at least one computer in their classrooms, a lesser number of teachers reported having 
computers in their classroom in the second year (85% to 63.6%). The teachers' responses to open-ended questions 
were consistent with this finding. When teachers were asked to describe any problems that they had with respect to 
computer integration in their classroom, the majority of those who responded listed a limited number of computers 
in their classrooms as a major problem. The discrepancy between the TRTs’ and the teachers’ reports as well as in the 
teachers' reports for the first year and the second year, can be explained by the teachers' lack of skills and knowledge 
about computers in the first year. Teachers lack of knowledge and skills might have been the reason that they were 
not able to distinguish incompatibility of their classroom computers in the first year, thereby reporting more 
computers in their classrooms. 
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Teachers were also asked to describe how they used computers in their classrooms. Interestingly enough, the 
results indicated that except for preparing hand-outs and printing materials, less teachers reported using computers in 
their classroom in the second years (see Table 4). The number of teachers who reported having educational software 
in their classroom also decreased (64.2% to 46.1%). The decrease in the usage of the classroom computer could be 
due to the incompatibility of the classroom computers with the computers in the lab, or due to the availability of the 
computer lab for classroom instruction. 

Usage of computers in the classroom varied slightly from one school to the next. In the first year the 
majority of the teachers indicated that they were mainly using computers for word-processing and subject-matter 
software. Enrichment/remediation and keyboarding ranked as the third and the fourth most common applications of 
computer usage in the classroom. However, in the second year, those teachers who were using their classroom 
computers reported that they were using their computers mainly for enrichment and/or remediation (17.1%), woni- 
processing (10.7%), and presentation/grade/individualized programs (10.7%). This result is consistent with the result 
of the closed-ended items which asked a similar question. The highest percentage of the classroom computer usage 
was for printing materials, enhancement of learning, and enrichment/remediation. The interview and observation 
data also revealed that when attending computer labs, students primarily used word-processing, games and subject- 
specific software for remediation. 

Teachers were also asked to rate their computer skills in order of strongest to weakest. The results woe 
similar for the first and second year. Teachers across different schools rated their word processing skills as the 
strongest computer skills and telecommunications as their weakest. When asked to explain why they had chosen 
some areas as their weakest areas, in the first year teachers listed factors such as lack of time and training. In the 
second year, in addition to lack of time and training, they also included factors such as lack of practice and 
experience in those skills, lack of confidence in their skills and less need for those skills. 

Teachers’ level of confidence in computer competency skills were also assessed. When teachers were asked 
to describe their confidence in computer competency skills at the beginning of the school year, 17.0% reported that 
they had all of the required computer competency skills in the first year. At the beginning of the second year only 
20.1% of teachers thought that they had all of the required computer competency skills. However, when asked to 
describe their computer competency skills at the end of the second year, almost half (47.2%) of teachers reported that 
they had confidence in all of the required computer competency skills. 

Discussion 

The purpose of the study was to assess the progress and the effectiveness of TRTs in their effort to assist 
classroom teachers to integrate computer technology into their curriculum and instruction. As the results indicate, 
TRTs were successful in establishing and maintaining a computer lab in their respective schools. They were also 
successful in providing regular training workshops and assisting teachers pass the computer competency tests 
mandated by the state. The improvement in the teachers’ level of confidence in computer skills and the increased 
number and usage of home computers by teachers can also be attributed to TRTs instructional support. 

However, based on the findings of the study, teachers did not show any major improvement in their attitude 
or usage of computers in their classroom instruction despite receiving regular training and technical support. This is 
contrary to the findings of the studies that suggest computer training can lead to a more positive attitude toward 
computers (e.g., Ernest & Lightfoot, 1986, Gressand & Loyd, 1985, Thompson, 1985). Furthermore, the study 
suggests that the number of teachers who were able to integrate technology into their curriculum and instruction 
remained unchanged. The slight improvement in teachers’ attitude can be explained by their lack of computer usage 
and computer integration in the curriculum. This is again inconsistent with the findings of Day and Scholl (1987) 
that teachers’ attitudes toward technology could be changed through organizational support and training. In the 
present study, TRTs as instructional and technical support were not successful in improving teachers’ attitude beyond 
a certain level over a two year-period. As the review of other studies suggests (Chin & Hortin, 1993), this 
inconsistency can be due to the teachers' need for ample time to acquire the knowledge, understanding and skills in 
instructional technology before they begin to use and integrate it into their curriculum. 

Consistent with numerous studies, the results of MANOVA tests in this study confirm that teachers who 
are currently using computers in their classrooms tend to like computers more, show more confidence in learning and 
using computers, and have less anxiety about computers. Furthermore, teachers who rate their computer skills high 
show a more positive attitude toward computers. Therefore, one can assert that there is a positive relationship 
between attitude and computer usage. Once teachers master the skills and begin using computer technology in their 
classroom and integrate it into their curriculum and instruction, their attitude will improve. 



Another important finding relates to the strategies that TRTs used in helping teachers integrate computers 
into their classrooms. Since not many teachers used or integrated computer technology into their classroom, it is 
hard to assess the effectiveness of the TRTs’ strategies in instructional support. The two most frequent strategies 
used by TRTs to help teachers acquire computer competencies were: making training a top priority at the school 
level and requiring teachers to take responsibility in learning and using computers. Anderson and Odden (1986) also 
reported the importance of making training a priority in order to enhance teachers' desire to commit to learn and to 
become successful in educational technology. 

Finally, the study suggests that despite the limited number of teachers who were able to use and integrate 
computers into their curriculum and instruction, TRTs made progress in their long-term goal. Teachers’ positive 
perception of TRTs and their role at the school combined with the improvement in teachers' computer competency 
skills across the schools are indicators of the TRTs' progress in their effort. 

Based on the findings of the study, it seems that if teachers are to integrate computers in their curriculum 
the following factors must be taken into account. 

1 . Organizational support plays an important role in any technological transformation within a school system. As 
the findings of this study suggest, organizational support must be provided at different level in order to assure 
the implementation of any technological innovation. The systems outside schools including districts, 
community and state can support the technological transformation by establishing requirements for computer 
integration. For example, in this study, the state mandated computer competency tests established the need for 
planning new goals and providing instructional and technical support. Without this organizational requirement, 
whether or not TRTs would experience any success in their efforts is not certain. 

2. Teachers needs ample time and training in order to acquire knowledge and understanding of what instructional 
technology can do for them. It seems that on-site training should continue for a long period of time if teachers 
are to integrate computer technology into their instruction and transform their curriculum. The training 
workshops must concentrate more on subject areas and grade levels. Teachers need to have a greater say in the 
nature of the activities and workshops topics. TRTs’ ongoing assistance needs to continue until teachers build 
a higher level of confidence and competence and are less intimated by the computer technology. 

3. Accessibility of computer technology in the classroom is as important as accessibility of computer labs. As 
teachers in this study indicated, technological availability in the school (computer labs) is not enough. 
Teachers also need to have access to computers and other current technologies in their classrooms or teachers’ 
work stations in order to begin using them. Although a centralized computer lab is very important in assisting 
teachers learn computer skills and in helping them use it to teach students, a classroom computer encourages a 
teacher to use it more often with student. Establishing a technology production center that houses the 
peripherals (e.g., scanner, camcorders, and digital cameras) for teachers is also important, especially when 
teachers begin using multimedia. 

4. Finally the extent to which TRTs are prepared to face the problems and provide immediate solutions and 
assistance is critical . A knowledgeable, well-trained and well prepared TRT who keeps himself/herself updated 
in instructional technology is an essential part of the present organizational support system for computer 
integration. There needs to be an on-job and or in-service training program for TRTs in both instructional 
technology and instructional design in order to keep them updated in their knowledge and skills. The field of 
instructional technology can provide leadership in this area. Time finally seems to be ripe for instructional 
technologists to offer their expertise to the public schools. 
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Table 1. Multivariate analysis of variance for teachers attitude toward computers school, usage of computer, 
previous experience and rate of previous and present computer skills in first year. 



Univariate F Ratio 

Multivariate Tests of Significance Factors measuring teachers’ attitude toward computers 



Source 


F Value 


1 


2 


3 


4 


5 


1. Attitude by school 


1.1 


1.5 


0.4 


1.6 


0.7 


1.3 


2. Attitude by usage of computer 


3.3* 


10.5** 


7.4** 


0.5 


1.8 


1.2 


3. Attitude by previous computer 


2 6 *** 


9 3*** 


0.8 


2.9* 


12 i*** 


8 . 2 *** 


experience 


4. Attitude by rate previous 


39 *** 


13 i*** 


0.8 


6 . 8 *** 


21 . 2 *** 


15.7*** 


computer skills 


5. Attitude by rate present ability 


5 3*** 


17.0*** 


0.4 


16.9*** 


37.1*** 


22 . 6 *** 



to use computer 



Note: Analyses are on (1) liking computers, (2) value for computer in education, (3) confidence 
learning computers, (4) confidence using computer, and (5) anxiety for computers. 

1 df= 5, 122 2 df= 1, 126 3 & 4 df= 4, 123 5 df= 4, 123 

* pc. 05 **p<01 ***pc.001 



Table 2. Multivariate analysis of variance for teacher’s attitude toward computers school, usage of computer, 
previous experience and rate of previous and present computer skills in second year. 

Univariate F Ratio 



Multivariate Tests of Significance Factors measuring teachers' attitude toward computers 



Source 


F Value 


1 


2 


3 


4 


5 


1 . Attitude by school 


1.0 


. 0.9 


0.8 


0.3 


0.9 


0.9 


2. Attitude by usage of computer 


1.9 


6.3** 


0.6 


3.4 


8 . 1 ** 


9.0** 


3. Attitude by previous computer 


4 i ** * 


8.5*** 


2.1 


6.3*** 


12 . 1 *** 


15.1*** 


experience 


4. Attitude by rate previous 


3.6*** 


10 i*** 


0.6 


8 . 6 *** 


14 4 *** 


16.7*** 


computer skills 


5. Attitude by rate present ability 


4 g*** 


16.5*** 


1.6 


18.8*** 


2i 4 *** 


23.2*** 



to use computer 



Note: Analyses are on (1) liking computers, (2) value for computer in education, (3) confidence 
learning computers, (4) confidence using computer, and (5) anxiety for computers. 

1 df= 5, 108 2 df= 1, 111 3 & 4 df= 4, 1085 df= 4, 108 

* pc. 05 **pc01 ***pc.001 
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Table 3. Cross tabulation and comparison of computer competency exams taken by teachers 
by school and by year. 



Schools 



Question 


4 


Year 


i 


2 


3 


4 


5 


6 






First 


(n=17) 


(n=17) 


(n= 1 7) 


(n=17) 


(n=17) 


(n=17) 






Second 


(n=24) 


(n=24) 


(n=24) 


(n=24) 


(n=24) 


(n=24) 


How many hours of 


First 


16.0 


17.0 


16.0 


17.0 


17.0 


17.0 


computer workshops 


Second 


16.9 


16.9 


16.9 


16.9 


16.9 


15.5* 


have completed? 


















Which of the 


following computer 














competency exams have you 














taken? 


Total 
















• Keyboarding 


38.1 


First 


14.3 


25.7 


8.6 


11.4 


11.4 


28.6 




87.6 


Second 


17.7 


17.7 


17.7 


16.9 


15.4 


14.6 


• Operating 


27.1 


First 


19.2 


34.6 


4.5 


13.6 


9.1 


13.6 




80.2 


Second 


19.2 


15.0 


20.0 


16.7 


15.8 


13.3 


• Word- 


28.4 


First 


22.7 


36.4 


4.5 


13.6 


9.1 


13.6 


processing 


84.8 


Second 


18.1 


16.5 


18.1 


17.3 


16.5 


13.4 


• Spreadsheet 


29.5 


First 


23.1 


30.8 


3.8 


7.7 


11.5 


23.1 




87.6 


Second 


17.7 


17.7 


17.7 


16.9 


14.6 


15.4 


• Data base 


25.0 


First 


23.8 


42.9 


0.0 


4.8 


9.0 


19.0 




81.1 


Second 


19.0 


15.7 


19.8 


17.4 


14.0 


14.0 



Number 1 to 6 represents different schools 

* Indicates results significant (chi-square = 6.7 with 15 degrees of freedom, pc.001) 
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Table 4. Comparison of means and standard deviations of teachers' different computer skills by school and by year. 



Schools 



Question Year 

First 

Second 


i 

(n=17) 
M (SD) 
(n= 24) 
M (SD) 


2 

(n=17) 
M (SD) 
(n= 24) 
M (SD) 


3 

(n=17) 
M (SD) 
(n= 24) 
M (SD) 


4 

(n=17) 
M (SD) 
(n=24) 
M (SD) 


5 

(n=17) 
M (SD) 
(n= 24) 
M (SD) 


6 

(n=17) 
M (SD) 
(n= 24) 
M (SD) 


List the following computers 
skills in order of your strongest 
areas to your weakest area (1= 
strongest and 9= weakest) 














• Terms and First 

operations Second 


2.7 (1.9) 

3.7 (2.2) 


4.4 (2.3) 
4.6 (2.0) 


3.0 (2.0) 
3.3 (2.2) 


3.4 (1.6) 
3.7 (1.7) 


3.8 (2.3) 
4.0 (1.9) 


4.6 (2.6) 
4.5 (2.2) 


• Keyboarding First 

Second 


3.6 (2.7) 
2.8 (2.6) 


2.2 (2.0) 
1.9 (1.2) 


2.3 (2.1) 
2.3 (1.9) 


1.7 (1.2) 
1.6 (.67) 


4.3(2.6) 
1.7 (.95) 


2.7 (2.5) 
3.2 (2.7) 


• Societal uses First 
Second 


5.8 (1.9) 

5.9 (1.7) 


5.5 (2.5) 
6.1 (2.0) 


5.5 (2.6) 
5.0 (2.5) 


6.4 (1.7) 
6.2 (2.2) 


6.4 (2.0) 

5.4 (2.4) 


4.3 (2.1) 
6.2 (2.0) 


• Ethics First 

Second 


5.6 (2.4) 
6.4 (2.2) 


5.5 (2.7) 
5.9 (2.7) 


5.2 (2.3) 
4.9 (2.7) 


5.7 (2.5) 
6.0 (2.4) 


6.4 (2.5) 
5.1 (2.6) 


3.6 (2.3) 

5.6 (3.0) 


• Word- First 

processing Second 


3.8 (2.4) 
2.4 (1.6) 


2.7 (2.2) 
2.1 (1.7) 


1.7 (1.0) 

2.7 (2.2) 


2.0 (.9) 
2.3 (1.1) 


3.1 (2.5) 

2.1 (1.9) 


3.6 (2.3) 

2.7 (1.9) 


• Data bases First 

Second 


6.0 (2.0) 
4.6 (2.0) 


5.2 (1.8) 
5.0 (2.2) 


4.5 (1.8) 
5.7 (2.3) 


6.0 (1.9) 
5.5 (2.1) 


5.6 (2.2) 

4.6 (2.4) 


6.0 (1.6) 
6.0 (2.5) 


• Spreadsheets First 
Second 


5.6 (2.1) 
4.4 (2.0) 


5.2 (1.8) 
5.1 (2.1) 


4.6 (2.7) 
5.0 (2.3) 


5.0 (2.3) 
5.8 (2.2) 


5.2 (2.4) 
4.8 (2.3) 


6.1 (1.6) 
5.0 (2.4) 


• Curriculum First 
software use Second 


5.6 (2.2) 
6.1 (1.6) 


5.7 (2.2) 
6.0 (1.8) 


5.0 (2.3) 
5.7 (2.5) 


5.7 (1.8) 
5.5 (1.9) 


4.6 (1.8) 
6.4 (2.4) 


5.1 (2.5) 
5.6 (2.4) 


• Telecomupting First 
Second 


8.0 (2.7) 
7.9 (2.0) 


8.2 (.8) 
8.0 (1.3) 


7.7 (2.4) 
6.4 (2.7) 


8.2 (1.3) 
7.4 (1.7) 


6.8 (2.6) 
7.0 (2.6) 


8.3 (1.2) 
7.6 (2.5) 





244 



Appendix A. Teachers ' Attitude Toward Computers— Mean and Standard Deviation Scores 







1994-1995 


1995-1996 


Liking 


Computers 


M 


M 






rsDi 


1SD1 


11. 


I like using computers. 


1.63 


1.69 






(.94) 


(.95) 


16. 


I like using computers in my school work. 


1.82 


1.97 






(102) 


(1.12) 


17. 


I wish I could use computers more frequently at the school. 


1.82 


1.97 






(.94) 


(109) 


19. 


Once I start to work with the computer, I would find it 


2.40 


2.51 




hard to stop. 


(1.14) 


(118) 


21. 


If a problem is left unsolved in a computer workshop or 


2.41 


2.51 




in class, I would continue to think about it afterwards. 


(1.10) 


(1.20) 


23. 


Teaching using computers would be very interesting. 


2.09 


2.30 






(107) 


(1.22) 


25. 


I look forward to using the computers at school. 


1.90 


1.94 






(.97) 


(101) 


30. 


When there is a problem with a computer program 


2.53 


2.61 




I can't immediately solve, I would stick with it until I 


(1.11) 


(1.15) 




have the answer. 






37. 


I think working with computers would be both enjoyable 


1.88 


2.00 




and stimulating. 


(.91) 


(111) 


58. 


Someday I will have a computer in my home. 


1.63 


1.84 






(.95) 


(116) 


2L 


Working with computers is boring. 


4.25 


4.23 






(91) 


(96) 


Total 




24.36 


25.57 


Value 


of Computer for Education 


M 


M 






(SD) 


(SD) 


14. 


I will use my knowledge of computers in many ways as 


1.88 


1.95 




a teacher. 


(1.03) 


(1.04) 


31. 


Learning about computers is a worthwhile and necessary 


1.58 


1.57 




subject for all teachers. 


(.90) 


(.86) 


33. 


It is important to know how to use computers in order to 


2.80 


2.49 




get any teaching position. 


(1.16) 


(1.21) 


43. 


Supplying every student with a microcomputer is a worthy 


2.12 


2.12 




educational objective. 


(1.12) 


(109) 


44. 


Teachers should demand that they be taught how to use 


2.09 


2.12 




microcomputers in their classrooms. * 


(.95) 


(109) 


45. 


Microcomputers will require learners to become active 


2.02 


2.11 




in their learning. 


(.96) 


(.99) 


48. 


If we do not use microcomputers in school instruction, 


2.60 


2.45 




our students will grow up illiterate and deprived of a 


(1.23) 


(1.21) 




basic skill. 






49. 


If my school district had the money, I would insist that 


2.18 


2.13 




they buy microcomputers in most every school subject. 


(1.07) 


(1.12) 


54. 


Computers can improve learning of higher-order skills. 


1.95 


2.01 






(.89) 


(.94) 




245 241 



59. 


Computers will improve education. 


1.70 


1.78 






(.90) 


(.92) 


57. 


Our country would be better off if there were no computers. 


4.42 


4.40 






(.89) 


(.93) 


4fL 


Microcomputer instruction will deny students the opportunity 


3.75 


3.64 




to reason with others. 


(1.03) 


(1.03) 


4L 


Using microcomputers as a teaching tool puts too much 


3.56 


3.41 




additional work on already overburdened teachers. 


(116) 


(1.14) 


5a 


Microcomputers will increase the amount of stress and 


3.51 


3.50 




anxiety teachers experience in schools. 


(1.10) 


(1.20) 


1L 


Microcomputers will decrease the amount of teacher-pupil 


3.71 


3.53 




interaction in schools. 


(1.05) 


(1.11) 


52, 


Microcomputers will isolate students from one another. 


3.82 


3.61 






(.91) 


(1.04) 


53, 


I object to all the attention being given to computer technology 


3.60 


3.50 




because it detracts from the real problems now faced by 


(1.17) 


(1.20) 




teachers. 






£L 


Computers will displace teachers. 


4.14 


4.20 






(.96) 


(.93) 


56, 


Computers will dehumanize teaching. 


3.96 


4.00 






(1.00) 


(1.10) 


Total 




55.39 


54.52 


Value 


of Computer for Society 


M 


M 






(SD) 


1SD) 


10. 


Knowing how to use computers is a worthwhile and 


1.29 


1.30 




necessary skill. 


(.71) 


(.62) 


41. 


I will probably need to know how to use a computer in 


1.63 


1.63 




my classroom. 


(.93) 


(.90) 


24, 


I don't expect to use computers in my classroom. 


4.37 


4.37 






(.91) 


(.90) 


42, 


Computers are gaining too much control over people’s 


3.41 


3.30 




lives. 


(1.20) 


(1.30) 


Total 




10.7 


10.6 


Confidence about Learning Computers 


M 


M 






(SD) 


(SD) 


12. 


I feel confident about my ability to learn about computers. 


1.78 


1.76 






(.96) 


(.96) 


34. 


I know that if I work hard to learn about computers, 


1.60 


1.66 




I can do well. 


(.71) 


(.82) 


Total 




3.38 


3.42 
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Confidence About Using Computers 


M 


M 






1SDV 


fSDl 


27. 


I feel comfortable using computers. 


2.11 


1.92 






(1.14) 


(1.03) 


40. 


I am able to do as well working with computers as most of 


1.98 


2.04 




my fellow teachers. 


(1.02) 


(113) 


Total 




4.09 


3.96 


Anxiety (or Lack of it) about Computers 


M 


M 






(SD) 


rsD) 


JJL 


Working with a computer would make me nervous. 


3.52 


3.73 






(1.31) 


(1.27) 


1L 


I get a sinking feeling when I think of trying to use a 


4.06 


4.15 




computer. 


(1.15) 


(111) 


20* 


Computers make me feel stupid. 


4.04 


4.14 






(1.15) 


1.10 


26* 


I'm not the type to do well with computers. 


4.13 


4.10 






(1.00) 


(1.07) 


22* 


Computers make me feel uncomfortable. 


3.93 


4.05 






(116) 


(113) 


20* 


Computers make me feel uneasy and confused. 


3.97 


4.10 






(1.11) 


(1.12) 


38* 


I think using a computer would be difficult for me. 


4.09 


4.17 






(1.04) 


(1.03) 


Total 




27.74 


27.95 


Perception about Gender-Appropriate of Computer Use 


M 


M 






(SD) 


fSDl 


10* 


Using a computer is more important for males than females. 


4.58 


4.56 






(.92) 


(.95) 


22* 


More men than women have the ability to become computer 


4.61 


4.53 




scientists. 


(.73) 


(.88) 


22* 


Using computers is more enjoyable for males than females. 


4.54 


4.53 






(.83) 


(.84) 


35. 


Females can do as well as males in learning about computers. 


1.47 


1.54 






(.93) 


(1.04) 


39. 


Working with computers is more for males than females. 


4.60 


4.56 






(.72) 


(.81) 


Total 




19.80 


19.72 



Note: For positively- worded statements l=strongly agree and for negatively-worded statements 5=strongly disagree. 
Negative items are underlined 
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Appendix B. Means and Standard Deviations 



Table 1. Descriptive statistics for teachers' attitude toward computer by six different schools in first year. 





School 


School 


School 


School 


School 


School 




1 


2 


3 


4 


5 


6 




(n=24) 


(n=24) 


(n=24) 


(n=24) 


(n=24) 


(n=24) 


Factors measured teachers’ 


M 


M 


M 


M 


M 


M 


attitude toward computers 


(SD) 


(SD) 


(SD) 


(SD) 


(SD) 


(SD) 


1 . Liking computers 


24.2 


25.6 


22.9 


25.6 


21.1 


24.2 




(4.8) 


(4.9) 


(5.7) 


(8.2) 


(6.5) 


(6.7) 


2. Value for computer in 


55.7 


55.4 


56.1 


56.5 


54.7 


54.2 


education 


(5.1) 


(4.0) 


(3.0) 


(4.4) 


(7.4) 


(10.3) 


3 . Confidence for learning 


3.6 


3.2 


3.1 


3.6 


2.7 


3.1 


computers 


(1.0) 


(1.3) 


(1.1) 


(1.7) 


(1.1) 


(1.1) 


4. Confidence for using 


4.1 


3.8 


3.4 


4.4 


3.8 


4.1 




(1.4) 


(1.7) 


(1.3) 


(2.1) 


(2.5) 


(2.1) 


5 . Anxiety for computers 


28.1 


28.5 


30.8 


26.5 


27.3 


27.6 




(4.8) 


(5.3) 


(4.3) 


(7.4) 


(7.3) 


(7.7) 


Table 2. Descriptive statistics for teachers' attitude toward computer by 


six different schools in second year. 




School 


School 


School 


School 


School 


School 




1 


2 


3 


4 


5 


6 




(n=21) 


(n=21) 


(n=21) 


(n=21) 


(n=21) 


(n=21) 


Factors measured teachers’ 


M 


M 


M 


M 


M 


M 


attitude toward computers 


(SD) 


(SD) 


(SD) 


(SD) 


(SD) 


(SD) 


1 . Liking computers 


23.7 


25.8 


23.8 


23.5 


22.8 


26.3 




(5.9) 


(6.4) 


(4.7) 


(5.4) 


(4.8) 


(9.5) 


2. Value for computer in 


53.7 


52.5 


56.7 


54.6 


53.7 


53.4 


education 


(4.7) 


(5.5) 


(4.3) 


(3.4) 


(11.2) 


(10.1) 


3 . Confidence for learning 


3.3 


3.4 


3.1 


3.0 


3.4 


3.4 


computers 


(1.5) 


(1.2) 


(1.0) 


(1.1) 


(1.5) 


(2.4) 


4 . Confidence for using 


3.6 


4.3 


3.6 


3.3 


3.5 


4.2 




(1.7) 


(19) 


(1.3) 


(1.2) 


(1.7) 


(2.5) 


5 . Anxiety for computers 


29.5 


27.1 


30.2 


29.3 


30.1 


27.2 




(5.9) 


(6.8) 


(4.4) 


(5.5) 


(5.3) 


(9.1) 



24.4 
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Table 3. Descriptive statistics for teachers' attitude toward computer by previous experience in operating computers 
in first year. 



0 experience Few days 1 to 6 months 7 month Over a year 



Factors measured teachers' 
attitude toward computers 


M 

(SD) 


M 

(SD) 


M 

(SD) 


M 

(SD) 


M 

(SD) 


1 . Liking computers 


21.8 


23.6 


24.0 


27.2 


30.8 




(5.3) 


(4.0) 


(4.6) 


(7.6) 


(6.4) 


2. Value for computer in 


54.9 


54.4 


57.0 


56.5 


57.0 


education 


(6.7) 


(4.8) 


(3.3) 


(3.9) 


(7.5) 


3 . Confidence for learning 


3.0 


3.6 


3.3 


3.7 


3.9 


computers 


(1.1) 


( 1 . 4 ) 


( 1 . 3 ) 


( 1 . 7 ) 


(1.1) 


4. Confidence for using 


3.3 


3.9 


3.8 


4.7 


6.4 


computers 


(1.6) 


( 1 . 3 ) 


(1.1) 


(2.0) 


(2.0) 


5 . Anxiety for computers 


29.9 


28.3 


28.8 


25.9 


20.8 


N=24 


(5.8) 


(3.4) 


(3.8) 


(6.2) 


( 7 . 1 ) 



Table 4. Descriptive statistics for teachers' attitude toward computer by previous experience in operating computers 
in second year. 





0 experience 


Few days 


1 to 6 months 


7 month 


Over a year 


Factors measured teachers' 
attitude toward computers 


M 

(SD) 


M 

(SD) 


M 

(SD) 


M 

(SD) 


M 

(SD) 


1. Liking computers 


22.8 


26.1 


29.5 


28.2 


36.7 




(4.5) 


(7.3) 


(6.0) 


(8.9) 


(11.6) 


2. Value for computer in 


54.6 


54.5 


55.6 


48.1 


57.3 


education 


(6.5) 


(5.6) 


(4.4) 


(9.8) 


(13.4) 


3 . Confidence for learning 


2.9 


3.8 


4.0 


4.3 


5.7 


computers 


(1.0) 


(1.4) 


(1.6) 


(2.2) 


(3.8) 


4 . Confidence for using 


3.2 


4.3 


4.9 


5.2 


8.0 


computers 


(1.3) 


(1.7) 


(1.7) 


(2.2) 


(2.0) 


5 . Anxiety for computers 


31.0 


27.8 


23.8 


19.0 


21.7 


to 


(4.2) 


(5.9) 


(7.6) 


(6.4) 


(12.7) 
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Table 5. Descriptive statistics for teachers' attitude toward computer by rate computer skills before school year in 
first year. 



Factors measured teachers' 
attitude toward computers 


None 

M 

(SD) 


Poor 

M 

(SD) 


Average 

M 

(SD) 


Good 

M 

(SD) 


Excellent 

M 

(SD) 


1 . Liking computers 


19.1 


19.3 


23.1 


26.7 


30.2 




(5.4) 


(3.8) 


(5.8) 


(5.9) 


(5.8) 


2. Value for computer in 


56.2 


54.3 


55.2 


55.8 


58.0 


education 


(3.2) 


(6.7) 


(7.3) 


(4.0) 


(6.9) 


3 . Confidence for learning 


2.3 


2.6 


3.3 


3.8 


3.8 


computers 


(1.0) 


(0.7) 


(1.1) 


(1.5) 


(1.3) 


4 . Confidence for using 


2.0 


2.6 


3.7 


5.1 


6.1 


computers 


(0.0) 


(10) 


(1.2) 


(1.9) 


(2.3) 


5 . Anxiety for computers 


34.4 


31.6 


29.2 


24.9 


20.7 


N= 24 


(1.3) 


(5.1) 


(4.7) 


(5.8) 


(7.4) 



Table 6. Descriptive statistics for teachers' attitude toward computer by rate computer skills before school year in 
second year. 



Factors measured teachers' 
attitude toward computers 


None 


Poor 


Average 


Good 


Excellent 


M 

(SD) 


M 

(SD) 


M 

(SD) 


M 

(SD) 


M 

(SD) 


1 . Liking computers 


19.3 


21.7 


25.5 


28.3 


33.0 




(4.3) 


(4.1) 


(4.6) 


(7.5) 


(14.0) 


2. Value for computer in 


53.9 


53.5 


55.6 


53.4 


52.0 


education 


(9.3) 


(7.0) 


(5.0) 


(7.7) 


(13.2) 


3 . Confidence for learning 


2.7 


2.7 


3.2 


4.3 


5.7 


computers 


(1.3) 


(0.8) 


(1.1) 


(1.7) 


(3.8) 


4. Confidence for using 


2.9 


2.7 


3.9 


5.3 


6.3 


computers 


(1.3) 


(1.0) 


(1.4) 


(1.8) 


(4.0) 


5 . Anxiety for computers 


33.6 


32.0 


28.7 


23.6 


16.3 




(3.4) 


(3.3) 


(5.0) 


(7.0) 


(11.4) 



N= 21 
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Table 7. Descriptive statistics for teachers' attitude toward computer by rate present ability to use computer in first 
year. 



None Poor Average Good Excellent 



Factors measured teachers’ M M M M M 



attitude toward computers 


(SD) 


(SD) 


(SD) 


(SD) 


(SD) 


1. Liking computers 


17.9 


21.5 


26.6 


30.4 


22.0 




(3.3) 


(4.6) 


(5.7) 


(6.3) 


(2.8) 


2. Value for computer in 


53.9 


55.5 


55.8 


56.3 


54.5 


education 


(7.4) 


(4.5) 


(7.2) 


(5.4) 


(3.5) 


3 . Confidence for learning 


2.2 


2.8 


3.6 


4.5 


2.0 


computers 


(0.4) 


(0.9) 


(11) 


(1.5) 


(0.0) 


4 . Confidence for using 


2.2 


3.0 


4.4 


6.6 


2.5 


computers 


(0.6) 


(1.1) 


(1.4) 


(1.9) 


(0.7) 


5 . Anxiety for computers 


32.8 


30.9 


27.0 


19.8 


34.5 




(4.9) 


(4.3) 


(4.6) 


(6.6) 


(.7) 



N=24 



Table 8. Descriptive statistics for teachers' attitude toward computer by rate present ability to use computer in second 
year. 





None 


Poor 


Average 


Good 


Excellent 


Factors measured teachers' 
attitude toward computers 


M 

(SD) 


M 

(SD) 


M 

(SD) 


M 

(SD) 


M 

(SD) 


1. Liking computers 


20.1 


23.0 


27.7 


30.3 


49.0 




(3.8) 


(4.6) 


(6.0) 


(6.9) 


(0.0) 


2. Value for computer in 


52.7 


55.0 


54.1 


51.2 


67.0 


education 


(9.8) 


(5.0) 


(7.1) 


(7.7) 


(0.0) 


3 . Confidence for learning 


2.5 


2.8 


4.1 


4.2 


10.0 


computers 


(1.0) 


(1.0) 


(1.5) 


(16) 


(0.0) 


4 . Confidence for using 


2.6 


3.2 


4.7 


6.5 


10.0 


computers 


(1.1) 


(1.2) 


(16) 


(2.1) 


(0.0) 


5 . Anxiety for computers 


33.0 


31.1 


25.1 


19.5 


7.0 


N=21 


(3.2) 


(4.1) 


(5.7) 


(8.6) 


(0.0) 
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Table 9. Descriptive statistics for teachers ' attitude toward computer by those who are currently using computers 
and those who are not currently using computers by year. 

Using computers Not Using computers 



Factors measured teachers' 
attitude toward computers 


94-95 
M (SD) 


95-96 
M (SD) 


94-95 
M (SD) 


95-96 
M (SD) 


1 . Liking computers 


23.1 (5.7) 


23.8 (6.1) 


28.2 (8.1) 


27.9 (6.0) 


2. Value for computer in 


54.9 (6.1) 


54.0 (7.1) 


59.0 (5.3) 


55.5 (5.8) 


education 


3 . Confidence for learning 


3.3 (1.2) 


3.2 (1.5) 


3.4 (1.7) 


3.9 (1.1) 


computers 


4 . Confidence for using 


3.8 (1.8) 


3.6 (1.7) 


4.4 (2.1) 


4.9 (1.9) 


computers 


5 . Anxiety for computers 


28.4 (6.1) 


29.6 (5.8) 


26.6 (7.8) 


24.7 (7.8) 



1994- 1995 (first year) N= 24 

1995- 1996 (second year) N= 21 
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A Critical Approach to Teaching Educational Technology 

Randall G. Nichols 
University of Cincinnati 



Toward Teaching Educational Technology 

In my fourth year of teaching high school English of all sorts, out of the blue one day in the spring of 
1978, I looked differently at the students. One was passing a note to another, two or three were nodding off, and 
maybe four or five were about to flunk American Literature. I thought to myself, “There must be a better way.” I 
resigned. 

That summer, Dr. Sara Games hired me as an instructor at Ohio State University (O.S.U.) to teach 
incoming Freshman who were very inexperienced writers. A year or so later, I knew I couldn’t stay at the position 
forever, and I was wondering what to do for a living. Another event out of the blue hit me; I remembered Sid Eboch 
and his 1975 or 1976 introductory educational technology course at O.S.U. He had been excited about satellites and 
computers, and I wondered if instructional technology didn’t offer an opportunity for me to be in the forefront of 
radically changing education in such a way that students wouldn’t be bored or intellectually bullied. So I enrolled in 
O.S.U. ’s Instructional Design and Technology doctoral program. My teachers, mentors, and acquaintances included 
John Belland, Chuck Czuri, Edgar Dale, Suzanne Damarin, Sara Games, Keith Hall, Kathy Irwin, Bemie Mehl, 
Holli Schauber, Bill Taylor, I. Keith Tyler, Bob Wagoner. 

Toward Resistance 

In my very first quarter in that program, in the fall quarter of 1979, Bill Taylor (for whom I was a teaching 
assistant) and I were talking at length and depth about the potential of educational technology. I was an enthusiastic 
TA and pronounced that microcomputers held great promise for making American schools more academically 
successful and humane (or words to that effect). He paused and quietly suggested that, “I’m not so sure that’s the 
case.” BAMM! I knew instantly and almost intuitively what he meant. Not only would technology do no such 
thing, but just the opposite might be a likely outcome. Schools could become even more repressive and boring 
places, where people write even more poorly or have no idea how to get corporations to stop convincing consumers 
to make bad money deals. This conversation, this moment, connected me with an intellectually and emotionally 
grounding truth. It also connected to my innate rebelliousness. It rang true and changed me forever, and the change 
was not frightening, even though I couldn’t say specifically what it meant for tomorrow. 

This moment connected me with a large, new world of issues, thinkers, and writers. I began reading the 
works of Michael Apple, Morris Berman, Hubert and Stuart Dreyfuss, Jacques Ellul, Lewis Mumford, Douglas 
Noble, Douglas Sloan, Manfred Stanley, Langdon Winner, and many others. Ted Nunan’s (1983) Countering 
Educational Design was a particularly eye-opening read. I watched films like My Dinner With Andre (George & 
Karp, 1982). And I spent much time talking with Bill about the basic nature of technology, whether or not 
technology could be autonomous, how instructional design was a technology, the ways schools generally seemed to 
be technological, the difference between educational and training, and how the microcomputer was not going to 
change education and learning except to more fully immerse people in a technopoly-to use Postman’s (1992) 
description. 

One of the best things about that time was that in about 1982, Bill and I and others began developing a 
course called “Technology, Society, and Schools,” which Bill is still teaching. Also, I started making a few 
presentations at the annual meetings of the American Educational Research Association (AERA), the Association for 
Educational Communications and Technology (AECT), the National Association for Science, Technology, and 
Society (NASTS), and the International Visual Literacy Association (TVLA). It was about at this time that several 
people began having what might be considered a Frankfurt School for critical theory of educational technology! We 
called these meetings the Leisure Time Institute (LTI), which was our way of poking fun at a faculty job description 
we had seen for a professor of leisure time studies. We thought that title was acutely indicative of the extent to 
which technology has seized people’s consciousness. LTI meetings were attended by Jane Johnsen, Jim Swartz, 
Kathy Irwin, their significant others, and assorted passersby. Seriously, these informal, very comfortable gatherings 
of the LTI helped me to formulate some of my basic understandings of technology and were very valuable as a result. 
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At the 1983 (if I remember correctly) AERA convention. I met Randy Koetting for the first time and heard 
him deliver a paper. I saw him formally mix broad-based philosophical critique with educational technology. I was 
quite impressed and encouraged. At about the same time, Bill and Jane Johnsen (1986) wrote “Resisting 
Technological Momentum” for a NSSE yearbook (Culbertson and Cunningham, 1986). I still consider this work to 
be ground-breaking in its critique and a foundational work for anyone interested in fully understanding the broad 
sweep of educational technology and how little we have pursued an understanding of it. 

Out of these prior experiences, I decided to write my first paper which would try to uncover all negative 
relations of educational technology, how they came to be so, and how we might begin to resist the negative 
elements. The paper was titled “Negative Aspects of Educational Technology in an Era of High Technology,” and I 
presented it at the annual meeting of The Association for Educational Communications and Technology in Atlanta in 
1987. (The paper became “Toward a Conscience,” and appeared in The Journal of Visual and Verbal Languaging in 
1987.) It was very postmodern, in that I didn’t use any particular theoretical basis except what was in my experience, 
and that was as much tacit as conscious-rational knowledge. What I mostly had at the time was my intuition and 
belief that I knew something wasn’t right. I worked truly from self/experience grounded theory, and it felt good. 

However, soon thereafter I started looking for more overt theoretical guidance. I dabbled a bit in 
existentialism at about this time, especially because I had come across William Barrett’s (1978) wonderful 
philosophical explanation of technology in The Illusion of Technique . Also, I certainly had heard about the work of 
Jurgen Habermas. I knew he was a leading philosopher in the field of critical theory and that he and people like him 
were interested in social justice and in critiquing rationality, so I read and studied his theories about communicative 
action (Habermas, 1984, 1987). One result of this was a paper called “Reconciling Educational Technology With 
The Life world: A Study of Habermas’ Theory of Communicative Action” (Nichols, 1991), a version of which was 
presented at an AECT conference and a version of which was published in Hlynka and Belland’s (1991) Paradigms 
regained: Uses of illuminative, semiotic and post structural criticism as a mode of inquiry in educational technology . 
At the same conference where I presented “Reconciling,” I also heard Mike Streibel’s absolutely brilliant presentation 
of some of the relations of instructional design to critical theory, a version (Streibel, 1991) of which was published 
elsewhere later. 

Another recurring theme in my work/life has been ethics. It occurred to me long ago that changing the way 
people think about technology would require more than rational, intellectual, fact-based positions. In fact, those 
positions, and dependent beliefs in them, are some of the features of human condition which are most responsible for 
the technological predicaments and dangers we are in. A somewhat more holistic, ethical worldview (a la Gregory and 
Mary Catherine Bateson, 1988; C. A. Bowers, 1993; His Holiness the Dalai Lama, 1994) will have to explain 
technology and move people toward changes. Addressing the moral and ethical elements of educational technology 
has always seemed like the natural approach to take. I’ve written about and presented at conferences on issues like the 
way educational computing is killing the earth because production of computers uses up the earth’s resources and 
because discarding used computer plastic and chips is polluting. 

I’ve made a few more presentations and written a few more papers and chapters that are critical of 
educational technology. For instance, I (Nichols & Allen-Brown, 1996) have a chapter on critical theory in the first 
Handbook of Research for Educational Communications and Technology (Jonassen, 1996). And today I maintain at 
least an affinity for aspects of what I’ll call emancipative theories and practices: feminisms, critical pedagogies, 
post-theory positions. I’m interested in fighting hegemonic groups in whatever their forms, from whatever positions 
I can (even if my personal fight is somewhat feeble). The language of today’s educational freedom fighters is often 
too unintelligible, and I don’t have the intellectual capacities of some of them, but I am always heartened by their 
egalitarian fervor, and opposition to those few people who are doing so much harm to the earth and its inhabitants. 

Being Resisted 

I have never gotten many accolades or even simple acknowledgements for my works. The greatest 
satisfaction comes very much from within myself and from the very few people who have actually discussed my 
work with me. The only other times my ideas seem to be noticed are, for example, during some presentations when 
people have walk out red-faced, when conference attenders seethe obviously, or when people write on the session 
appraisal that “This is the worst presentation ever at AECT [a direct quotation]. It takes ‘politically correct’ to new 
heights of ridiculousness” [or words to that effect]. 

Only once has anyone gotten mad and confronted me openly over critical issues. This happened in a large- 
group meeting of professors in several disciplines from around my own college. I wondered aloud if some of our 
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courses didn’t train new teachers in such a technical way that when they got into the first day of the first class, and 
the lesson plan started to fall apart, they might be somewhat paralytic instead of quickly and professionally 
improvisational. “BULL SHIT” came screaming back at me from a professor who has the correct, indisputable 
answer to everything and feels it is his God-given duty to tell you the truth in no uncertain terms. Of course, I 
continue to maintain an interest in b.s.! 

Learners and Other Colleagues 

I am grateful that most of the time a student comes along and reminds me that I’m far more interested in 
getting students to see critically for themselves the content of school and the personal, social, and cultural 
ramifications of it. I am particularly indebted to Lauryne Alexis-Boyd, who has helped more than anyone to keep me 
on the roads toward emancipative educational knowledge, and Jenny Bishop, who, far more than anyone, told me the 
most critical thing about my teaching-that I was being overbearing and repressive to students who really wanted to 
try answering my questions about technology. I thank other people whom I have forgotten unintentionally. It’s not 
surprising that I can’t remember others, given my disposition to be alone and to think more about ideas than about 
people. And to all those students who seriously question my academics, I say, “Thank you.” If only for a second, 
you remind me to stay honest, just, and caring. I even thank those who threaten me somehow and for no good 
reason-like the high school boy who said he was going to kill me. I always need to be reminded that some people 
will fight a bad school system even if they don’t know why they are doing so and even if it means murder. 

I am grateful, also, that Rhonda Robinson, at Northern Illinois University, has encouraged me to write for 
various publications and to present at several conferences. She has had lots to offer many people by way of critical 
thinking about educational technology. To give just one example, she edited a special issue of The Journal of 
Thought (Robinson, 1990) to bring together some of the few works which examine our technology from historical, 
philosophical, and social views. I highly recommend its analyses to you. For me, though, Rhonda’s most notable 
contribution has been the times she has encouraged me to keep going ahead personally and professionally when the 
world seemed bleak. In fact, this paper wouldn’t exist if it weren’t for Rhonda. 

Of many colleagues at the university where I work, I have little to say that is encouraging. Too many 
university people and the institutions of which I am aware are good at attempting to crush enthusiasm, 
thoughtfulness, and self-worth. Sometimes this crushing occurs because of the technological aspects of institutions- 
aspects such as bureaucracy. For the most part though, many, many professors and others are mind-bullies who 
force themselves on others. (I attribute this bullying to deep ego fears they have covered with denial and academic 
degrees. They forget they have these fears, so they forget what it’s like to be emotionally and intellectually bullied.) 
Some of those who have been bullied have become hardened or dulled by school experiences, and they seem not to be 
hurt by the power of bullies. Others are capable of ignoring the bullying. A very few others rally against it. A 
substantial number are appalled and disheartened, even distressed, by it. I usually am distressed. American schools 
generally seem hardly to teach people to be curious, conversational, caring, or democratic. Of course, schools are not 
nearly totally to blame for this condition. Capitalism and the cult of sci-tech are more to blame. 

More Explicit Pedagogy and Curriculum 

There may be no more important reason for school than to help learners understand why they are in school 
and doing school work. This helping is the heart of my form of critical theory. Take the kid in the back of the room 
who leans against the wall with his (usually a male) arms folded. He challenges, “Why do we gotta learn this stuff?” 
When we can explain why to his satisfaction, schools will change, teachers will change, and he’ll be educated. 

To me, helping students to understand the purposes of their education means being as democratic as 
possible, which, in turn, means student responsibility for self and curriculum. I always do things to get students to 
take over, even if it means asking students to choose several of the class conversations, the course topics, the course 
goals, and the means of reaching their goals. For instance, there are several times in a course when I’ll tell learners 
I’m talking too much and they should take over. Sometimes, when I’m at a conference for instance, I’ll turn the 
entire class session over to them and ask them to record it for me to hear/watch later. It’s fascinating to see them 
“play to me” and struggle to converse with one another in these recordings. They must stay within my goals for 
them, but there is lots of leeway within the goals. 

I try always to include, if only by way of several sly asides, course components in which learners and I look 
critically (i.e., in a balanced fashion) at the full spectrum of relations associated with education and technology. The 
first course in which I did this was the one I helped Bill Taylor develop: “Technology, Society, and Schools.” But 



even in courses which are not obviously about critical issues, I work a little bit of critical demeanor into the course. 
For example, in an introductory, undergraduate, education course about school curricula, the course books (e.g., 
Slattery, 1995, p. 172) usually describe many frameworks for curriculum development as “technical.” I take a minute 
to explain technology from this point of view and indicate how much of education is technological-teaching 
methods, for instance. 

My predominant effort at helping learners to become technologically literate comes in the course 
“Educational Technology: Critical Perspectives,” the syllabus for which is attached to this paper. I first introduced 
and taught this course at U.C. in 1986, I’ve taught it every year but one since then, and it is now a required course 
for our masters degree in Instructional Design and Technology. A version of it may become a course which fulfills a 
general education requirement at U.C. 

You’ll notice that the texts listed on the current Critical Perspectives syllabus are Technopolv (Postman, 
1992), PoaULQlthQJ S Q y l (Barrett, 1987), and School’s Out (Perelman. 1992). Postman’s work gives a very readable 
history of technology and its relations to degrading culture and communication. Barrett’s work is a great summary of 
several philosophic views as they relate to technology and the forms of tyranny which often accompany it. For me, 
Perelman makes about the strongest, most enthusiastic argument for technologizing, capitalizing, and otherwise 
changing schools and learning. It’s fun to see students who, at the end of only six or seven weeks in the course, can 
spot, understand, and begin to resist the troubling and dangerous (and, in some ways, common) views espoused by 
the likes of Perelman. We started the course this year by watching After the Warming (Sattin & Slee, 1990) in 
which James Burke shows a history of technology and weather, and he uses scientific data to convince viewers that 
many people than would otherwise are going to suffer and die as a result of the global warming we have already 
created. I like to start the term on an emotional as well as intellectual level for the learners! 

Over the years I’ve also used the following texts, films, and articles (and many more, besides): 



Books 

Teachers and Texts (Apple, 1986) 

The Jobless Future (Aronowitz & DiFazio, 1994) 

Mind and Nature (Bateson, 1979) 

Education, Cultural Myths, and the Ecological Crisis (Bowers, 1993) 

The Cultural Dimensions of Educational Computing (Bowers, 1988) 

Children & Computers in School (Collis. et al, 1996) (especially, p. 64, “Our conclusion....) 

Paradigms Regained (Hlynka & Belland, 1991). 

Technology and The Future of Schooling (Kerr, 1996) 

Rethinking Media Literacy (McLaren, Hammer, Sholle, & Reilly, 1995) 

Computers in Education (Muffoletto & Nelson Knupfer, 1993) 

Time Wars (Rifkin. 1987^ 

The Whale and The Reactor (Winner. 1986) 

Understanding Computers and Cognition (Winograd & Flores, 1987) 

Video & Films 

Practically anything shown on TV is instructive of the ways technology threatens our culture, ourselves. 
After the Warming (Sattin & Slee, 1990) 

Mv Dinner with Andre (George & Karp, 1982) 

The Tribe That Time Forgot (Miles. 1994) 

The Public Mind (Public Affairs Television, 1989) 

The Virtual Wasteland: Visions of Heaven and Hell: Information Technology and The Future (Harrison. 1995) 
Promotional/educational videos about education and “exciting” technologies, e.g., AT&T and Ameritech tapes 

Articles 

Critical Theory and Educational Technology (Nichols & Allen-Brown, 1996) 

Mad Rushes Into The Future: The Overselling of Educational Technology (Noble, 

Equity and Computers in The Schools: A Decade of Research (Sutton, 1991) 

Resisting Technological Momentum (Taylor & Johnsen, 1986) 



1996) 



More Pedagogy 

Beyond trying to help students understand why they are doing school work and helping students to be 
conscious of schooling and its social, political, cultural, and ecological meanings, other more specific teaching and 
learning approaches have guided me. 

For many learners, it is good enough just reading, writing, watching, and talking about texts and 
films/videos which demonstrate the problems of technology. These are newer and deeper issues than some people 
have encountered, so they usually are attentive, especially since many of them already have a good store of less 
conscious or considered experience which tells them that technology is problematical. They know how much a 
broken car costs. They know the difficulties that layers of technology often add to their school tasks. Almost all the 
time, for all of us, putting on a computer-generated and presented demonstration is more demanding (and not any 
more likely to succeed) than talking about it or putting it on a chalkboard or showing it on the old overhead 
projector. Because learners have these experiences, it is easier for them to watch an episode of video-based instruction 
and critique it. We can watch a 1980s instructional video which shows adolescents delivering newspapers as a way to 
teach youngsters how to use mental imaging and chunking for memorizing. My learners dig into the “hidden” 
objectives within the overt message. We find that besides memory, viewers learn uncritical acceptance of news, 
capitalism, police, punctuality, and so on. 

In the last few years, I’ve occasionally been requiring something more active, political, physical~an “action 
project.” Often these projects take students into the community beyond our university walls. Some students have 
begun paper or aluminum recycling projects back at their schools. One former student actually includes an 
examination of the negative effects of educational technology in her school’s computer curriculum and technology 
infusion. Similarly, a team of learners in a course last fall critiqued typical instructional design by erecting a block 
building on video as they talked about a typical design process-and then they tore it down with a great swing of an 
arm to show how ID is somewhat flawed. They want me to send the tape to the authors of the design text we used! 
One teacher was stunned to find Bowers’ (1993) three deadly cultural assumptions (progress, individualism, 
rationality) hidden in the texts about the environment her elementary students were using. My learners’ project topics 
usually can be on an issue of their choosing, as long as they stay within some broad parameters regarding course 
focus, coherence and mechanical correctness of the written documents, usefulness to someone outside of class, and so 
on. 

So What? 

I’ve always tried to be a teacher who understands the feelings of students and who is serious about teaching 
the subject matter at hand (even if I sometimes do so in a light-hearted fashion). But I admit that I’ve often fallen 
largely into the same role of controller that most teachers get into as a result of the culture of teaching, which often 
forces us to make students sit down, shut up, follow directions, and forget about why we are doing what we do in 
class. (Yes, I know schools teach other, more positive, behaviors and attitudes.) 

Moreover, judging by the continued rushing flood of calls by the likes of Vice-president Gore and Ohio’s 
Governor Voinovich for more educational technology, to say nothing of many professional education organizations 
boosting technology, what I do to increase technological literacy and resistance is too rarely successful. At an IVLA 
Conference in Pittsburgh in the early 1990s, a producer for PBS science- and technology-related programs told a large 
crowd that he could not do a program in which the problems of technology were addressed because there simply is no 
foundation, government, or private money to be found to support that kind of effort! 

More locally, only rarely have any of my students substantially picked up the challenge of resisting and 
thinking fairly about our technology. Yes, I’m sure I’ve instilled a slightly different outlook and an occasional 
questioning attitude in them, but I know of no one who’s maintained a vigorous, skeptical, and balanced stance 
toward educational technology, a stance which expresses itself in constant, noticeable, meaningful activity. Someone 
I am unaware of may be doing this, but very little political or economic action by my learners has ever taken place, 
that I know of. Yes, several learners have published balanced critiques of technology in sites ranging from school 
newsletters to regional technology conference proceedings to international journals for nuclear medicine, but no one 
has had an epiphany, a changed way of life. 

This is why one good research/clearinghouse need is for full-scale tracking of the kinds of disappointments 
and dangers educational technology is related to: classism, racism, ecological destruction and, changes in beliefs 
about the goodness of technology (and science?) in people’s lived experiences, not just in what they say they believe 
in. Once, at an international meeting of technology professionals, I recommended the establishment of an Office of 
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Educational Technology Assessment that would track, among other things, the negative influences of our 
technology. No one responded to this. Apparently, no one thought it is a good idea. 

Today and Tomarrow 

My agenda in the near future is uncertain. That is alright because I am pretty sure that “Those whom the 
gods would make mad, they first give knowledge of the future.” 

I am beginning to do a little writing about the relations of educational technology to democracy-noting for 
instance how technology such as television can diminish our democratic choices (Public Affairs Television, 1989). I 
am currently working in our new Cincinnati Initiative for Teacher Eduction (CITE) professional 
development/practice schools, and the opportunities to examine and encourage democracy and literacy about 
technologies in these settings is quite exciting. Some of our interns are feeling the stultification of the technologies 
of school rules and state proficiency exams, and I try to discuss this in every venue I can. 

Further, only now am I considering writing fully and openly about a direction to which I alluded earlier. No 
problem of schools or societies or ecologies will be solved using strictly rational-technical approaches (which are the 
approaches we seem to use today. This is called “the technological fix.”). Instead, one or two things may have to 
happen. Either some form of holistic thinking and/or a form of cathartic whole-person change is necessary for us and 
the earth to get out of the social and ecological dangers that are besetting us. This is to say that we must find 
somewhat conscious and conscientious ways in our social structures, in schools particularly, to meld the sacred with 
the profane, the overt with the tacit, the rational with the irrational, the school curriculum with the rest of the world 
(Dewey, 1966; Moffett, 1994; Orr, 1992). These kinds of mergers may become wide and deep enough that humans 
overcome our “progressive,” technological, capitalistic, controlling, and short-sighted natures and live a more 
balanced mental, emotional, and ecological existence. However, my examination of existence, how humans live, and 
our current embracing of technology, consumerism, and so forth, tells me that we will not change our existence 
until widespread human deaths force us to the indisputable realization that far too many people are going to die 
needlessly unless we change our philosophy about technology and its human meanings— unless we have a lifeworld 
catharsis. 

Also, I always turn to some form of spirituality in trying to understand my work and all of my life. Today, 
I hope that I begin taking action to see what Tibetan notions of spirituality and self-change mean to me and my 
work. And of course I’ll always fish, even if I don’t always catch fish, and even if I fish only in my imagination. 

Conclusion 

It is impossible to overstate the impact individuals and unplanned moments have had on my career/life. My 
knowledge acquisition and teachable moments have not been planned for the most part. Unlike what we wish for 
with technology, learning is not predictable! 

I also know how little I’ve done professionally-mostly because of my personal makeup and shortcomings, 
but also partly because so many people see so little to get worked up about in terms of dangerous educational 
technologies and other philosophies. 

All I have now may be the knowledge that something could be better for most people’s existences. 
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EDUCATIONAL TECHNOLOGY: CRITICAL PERSPECTIVES (18-214-774) 

Winter, 1997. Dr. Nichols, 624 TC. 556-3577. 

Texts: Technopoly (Postman, 1992), Death of the Soul (Barrett, 1986), School's Out (Perelman, 1992) 

week topic / activity / due 

1/9 Problems of Technology: After the Warming. 

Assignment: Begin to identify & solve a problem with technology. 

1/16 Foundations of Technology 

Characterize Technology , Educational Technology. 

The Tribe That Time Forgot — Technology Encroachment 
Due: Taylor & Johnsen Article-Discuss It 

1/23 History & Cultural Conditions of Technology 

Discuss Technopoly. 

Due: One Page Summary of Technovoh 

1/30 Philosophy of Technology 

Discuss Death of the Soul , chapters 1-5. 

2/6 Philosophy of Technology 

Discuss Your Summaries 

Due: Two Page Summary of all of Death of the Soul 
(Pass out article(s) for next week’s discussion.) 

2/13 Learner Led Discussion of Educational Technology: 

Philosophic/Cultural/Ecological Perspectives 

2/20 Tell (via video?) Nichols the upshot of last week’s discussion 
The Future of Educational Technology 
Discuss School's Out 

2/27 Watch/Discuss Perelman vs. Postman Tape. 

Bring and Share “Found” Materials 

3/5 Due: Outline of Your Plans For The Future 

3/13 Submit To Me Your Article To Be Submitted Foi^Publication (w/envelope). 

(Or turn in alternative assignment.) Discuss article with class. 

GRADED ASSIGNMENTS (Total Scale: 100-91 points = A; 86-90 = B; 81-85 = C; 76-80 = D) 

In addition to what’s described for each assignment, I will grade all work based on: 

•Use of references to the course work and other scholarly work, 

•Coherence/logic, and 
•Mechanical quality (good grammar). 

One Page Summary of Technopoly (25 pts) 

In one well-written page, tell the following about the book: 

•a general description of the whole book 
•the main thesis or theses 

•the main way(s) one (or more) of the theses is sensible 
•the main way(s) one (or more) of the theses is senseless 

This one page is to be single spaced , characters not larger than 12 point, margins not wider than one inch. Conform 
to APA style. 
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T wq Pa g g $u m m?W -& LDeath of the Soul (25 pts) 

In two well-written pages, tell the following about the book: 

•a general description of the whole book 
•the main thesis or theses 

•the main way(s) one (or more) of the theses is sensible 
•the main way(s) one (or more) of the theses is senseless 

This is to be single spaced, characters not larger than 12 point, margins not wider than one inch. Conform to APA 
style. 

Found Materials (5 pts) 

Bring in, on the date indicated on the syllabus, some material that you believe is pertinent to the course 
ideas (a material I might consider using the next time the course is offered?): a book, film, computer program, etc. 

XmLBmisSsiLlM (10 pts) 

Turn in a written, short sentence outline that tells what will you do in relation to educational technology as a result 
of this course? The outline has these parts: 1. Current Conditions With Educational Technology, 2. How 
Conditions Got This Way, 3. Your Plans For Dealing With The Conditions, and 5. Why The Plan’s Good. You’ll 
present this in class. 

Article To Be Submitted For Publication (35 pts) 

Using whatever format the publication requires, write an article to be submitted to that publication. At the 
end of the term, you’ll give me the article and an envelope addressed to the publication. Initially, I’ll read the article 
as if I were reading it in a journal (not as if I were an editor). Then I’ll decide if it’s quality is good enough to send 
to the publication. If it’s good enough, you get 35 points, and I send it; otherwise, you get 25 points, and I don’t 
mail it. I suggest that you form reading/editing peer groups or partnerships so that before you hand the article to 
me, it has been edited with a strict eye by one or more class members. Of course, you read their work. Whether or 
not it is published makes no difference to your grade. 

The publication you submit to is up to you; just make it an educational publication. 

The topic is up to you; just make it about education and technology. 

(Alternative) Survey: What Do Your Students or Teachers Know About Technology ? (35 pts) 

Create and apply a survey to find out how much some teachers /and or students know about: 

•The meanings of technology and educational technology. 

•The pervasiveness of technology generally in their lives, including the process and/or intellectual kinds of 
technology. 

•The pervasiveness of educational technology generally in their lives, including both media/hardware and the process 
and/or intellectual kinds of technology. 

•The actual and potential problems with educational technology. 

Investigate either students or teachers. Find at least ten students or teachers to take the survey. Use any age 
level, though high school age and older would be best, maybe. 

Write a brief summary of your findings. Include at least: background of survey, n, means (if appropriate), 
highlights of the findings, and your conclusions and recommendations for the relations of people to technology and 
educational technology. Use APA style. 

Note that I may somehow use some version of your ideas , questions , or findings in my future work. I 
don’t know how , and if I use your work in any recognizable way, I will attribute it to you. I have always wanted to 
do something like this and haven’t. Your work may help me get unstuck. If this is a problem for you, please let me 
know. Thanks. 
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Instructional Technology Benchmarks for Teacher Preparation 
Programs and K-12 School Districts 



Pamela Taylor Northrup 
University of West Florida 



Introduction 

During the past ten years, between 300,000 and 400,000 computers have been added annually to school 
inventories across the United States. An estimated 6 million computers are now in K-12 schools, with the number 
of computer purchases expected to increase annually during the next decade. Currently, seventy-five percent of all 
schools have access to telecommunications through local or wide-area networks; 35% have direct access to the 
Internet (U.S. Congress, 1995). Yet, neither teaching nor learning with technology is widespread (NEA, 1993; 
SERVE, 1993). 

As a result of inadequate preservice (Beaver, 1990; Brooks & Kopp, 1989) and inservice training (Shaw, 
1995), teachers are not being prepared to use technology. Researchers concluded that teachers are being inadequately 
prepared to use instructional technology, and consequently are unable to effectively integrate technology into 
classroom teaching practices (U.S. Congress, 1995). 

Preservice Teacher Preparation 

Currently, fewer than 40% of colleges, schools, and departments of education nationally require a single 
course in instructional technology with only 18 states requiring technology training for all teachers seeking 
certification (Anderson, 1994). For those colleges, departments, and schools of education requiring at least one 
course in instructional technology, many barriers exist. Lack of equipment, current software, and ongoing budgets 
for maintenance and upgrades continue to be problematic (Northrup & Little, 1996). Some argue that teaching a 
single course in instructional technology may not be the right approach (Roblyer, 1994). Many times a single 
survey course will focus on equipment and software use without making connections to curriculum integration. Few 
colleges and schools of education are beginning to integrate instructional technology applications within methods 
courses (White, 1991) in lieu of a survey course. Theoretically, this approach will model the integration and use of 
instructional technology within specific subject areas and grade levels. Regrettably, many methods faculty do not 
have the range of skills required nor the time in a single semester to teach students the hardware and software basics 
along with the conceptual understanding of how technology can be integrated. White (1991) proposes a three step 
model for technology teacher preparation: (1) Train teachers on what the computer (technology) can do; (2) Train 
teachers what learners can do with technology; and (3) Emphasize the new role of the teacher (in a technology-rich 
environment). 

K-12 Teacher Inservice Training 

School districts continue to struggle with technology inservice training for teachers. Parker (1991) 
indicates that both new and veteran teachers require the same amount of instructional technology training; college and 
schools of education are not preparing teachers to use technology in the classroom. Most teachers agree, with 59% 
of those surveyed reporting that their training was inadequate to prepare them to use successfully technology in the 
classroom (SERVE, 1993). In a 1988 study conducted by the Office of Technology Assessment, only one-third of 
K-12 teachers have had 10 or more hours of computer training. A 1996 study suggests that not much has changed 
with 36% receiving up to 10 hours of computer training (Northrup, Shaw, & Rasmussen, in press). Staff 
development research on instructional technology suggests that teachers should be trained to use technology in a 
hands-on setting with curriculum specific applications. Following should be ongoing support (U.S. Congress, 
1994). Glenn and Carrier (1989) propose three steps to effective inservice technology training: (1) propose 

examples of how effective teachers use technology; (2) be aware of significant anxiety concerns of teachers; and (3) 
assign peer teachers and model effective uses of technology at the school site. 
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Instructional Technology Guidelines 

National and state instructional technology guidelines are beginning to emerge that identify competencies 
required for graduates of preservice programs, for practicing teachers, and for students. At the national level, the 
International Society for Technology in Education (ISTE) has developed Curriculum Guidelines for Accreditation of 
Educational Computing and Technology Programs which have been adopted by NCATE as the instructional 
technology guidelines for teacher preparation programs. In addition, NCATE has standards in other areas reflecting 
exemplary technology use in all areas of the unit. For example, under the qualifications for professional education 
faculty, faculty must model the integration of computers and technology in their fields of specialization. With 18 
states requiring instructional technology training in colleges and departments of education, specific state requirements 
have been identified. In Florida, the Preprofessional Accomplished Practices and the Accomplished Practices for 
Technology for Technology serve as the accountability measure for technological competence at the end of a teacher 
preparation program (Preprofessional Accomplished Practices) and again at the fifth year of teaching (Accomplished 
Practices). Both serve as benchmarks for teacher performance. 

In Florida, some school districts are preparing for the implementation of Florida’s Sunshine State Standards 
(1996) by training teachers on the standards/benchmarks set for students by grade level cluster (P-2; 3-5; 6-8; and 9- 
12) and by subject. One mid-sized district in Northwest Florida is establishing staff development competencies for 
instructional technology based completely on the competencies identified for students in Florida’s Sunshine State 
Standards. 

“K-12 schools and preservice teacher education programs must make significant investments in faculty 
development to realize the full potential of technology. A need exists to identify specific benchmarks defining what 
preservice and practicing teachers should know and be able to do and how technology should be infused into the 
preparation of preservice teachers. Given the critical role of technology in K-12 education and the social and 
economic future of the nation, preparing teachers to effectively use instructional technology in the classroom is 
critical” (Northrup & Little, 1996, p. 213). The purpose of this study is to establish instructional technology 
benchmarks for K-12 and teacher preparation programs in colleges, departments, and schools of education. 

Method 

To establish benchmarks for teacher preparation programs and for K-12 schools reflecting the preservice and 
inservice training needs, four analysis approaches were conducted and data were triangulated using a qualitative 
pattern-matching approach. The analyses included: (a) a critical review of prominent guidelines [both National and 
State] (b) a key informant study; (c) an instructional technology survey; and (d) a review of instructional technology 
literature. 

Guidelines reviewed include: (a) The National Council for the Accreditation of Teacher Education; (b) 
International Society for Technology in Education’s Educational Computing guidelines; (c) Florida’s Sunshine State 
Standards; and(e) Florida’s Accomplished Practices for Instructional Technology. The guidelines were synthesized 
into a usable matrix to identify commonalities. The guidelines were then clustered by topical area (computer basics, 
telecommunications, integration, instructional design, etc.) and by order of presentation. 

Key Informant Meetings were conducted with educators from the surrounding school districts, a medium and 
a small district. The key informant meetings were composed of teachers representing all grades, school-based media 
specialists, special education educators, school-based administrators, district-level administrators, and the Director of 
Technology for each district. In each key informant meeting, participants were asked to respond to several questions 
regarding their level of instructional technology use in the school and district, the projected use in the future, and 
their concepts of what the university should do to prepare preservice and inservice teachers for the integration and use 
of instructional technology. 

Instructional Technology Teacher Preparation Survey . A 25- item survey was administered to 300 K-12 
teachers and administrators in the surrounding school districts to further reinforce the outcome of the key informant 
meetings. Educators responded to specific areas of instructional technology use and indicated which technologies 
must be taught in preservice teacher preparation programs. Additional demographic information was collected to 
include: grade level/subject taught; number of computers in the classroom; number of computers available in the lab; 
numbers of inservice hours attended; and percentage of time engaged in technology use for both administrative and 
classroom use. 

Review of the Literature. A critical review of the instructional technology literature was conducted to 
establish current trends in instructional technology and uses in public education. 
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Results 

Data were analyzed using a qualitative approach clustering key concepts into categories through a pattern 
matching approach. Five critical areas emerged with each category in this sytem representing key issues for K-12 
and teacher preparation: (1) State-of-the-Art Laboratories, (2) Preservice Teacher Preparation, (3) Graduate-level 
Preparation, (4) Higher Education Faculty or Teacher Peers as Models; and (5) Inservice and Outreach. The analysis 
revealed that all areas were critical for preparing preservice and inservice teachers to become leaders in instructional 
technology. Benchmarks were established for all five areas. Currently, the benchmarks are being used at the 
University of West Florida in preservice teacher training, in a graduate program in instructional technology, in 
faculty development, in outreach, and in upgrading equipment for the laboratories. 

State-of-the-Art Laboratories 

Laboratories were directly addressed in the NCATE Standards and in the key informant meetings. 
Technology-using K-12 teachers provided examples of their classroom and school infrastructure which included 3-6 
networked computers per classroom; networked computer labs; and ITV production studios. Teachers indicated that 
they would be unable to integrate technology effectively without the use of computers that could run current versions 
of software required. Additionally, in key informant meetings, school administrators suggested that the purchase of 
new computers must be driven by a set district standards for new equipment acquisition. In a review of the research, 
industry standards adopted by software developers such as IBM’s Educational Software Division (formerly EduQuest) 
and Microsoft set a target for software development, based on the current industry standard. Finally, the Secretary’s 
Commission on Achieving Necessary Skills (SCANS) competencies should be a consideration in any state-of-the-art 
laboratory facility. According to SCANS, students are not being prepared to enter the workforce. Student exposure 
to instructional technology ranges from 25 minutes to 1 hour per week. The exposure to a variety of platforms and 
experiences is directly related to the level of comfort a teacher has with technology. Laboratories should maintain 
multiple platforms and peripherals. 

Benchmark: State-of-the-Art Laboratories 



• “Multiple platforms should exist including Macintosh and IBM. (Power Macintoshes may be 
most appropriate). 

• Adhere to the current industry standard as closely as possible. For example, if software is no 
longer available for System 6.07 for the Macintosh and all software of interest requires a minimum 
of 8 mb of RAM, do not purchase a Macintosh Plus. 

• A minimal number of Apples (Apple lie or Ilgs) should be maintained as many schools are still 
operating with this configuration. 

• One or more multimedia development stations with memory and hard disk capability to capture and 
playback video and sounds should be provided. 

• Several stations with connectivity to the Internet, preferably graphic-based browser software such 
as NetScape should be available for student and faculty research. 

• Modems should be no less than 28.8 bps. 

• Point-to-point and multi-point desktop videoconferencing should be accessible to students through 
the LAN and to the region via a MAN or WAN. 

• CD-ROM at a minimum of 4X should be included on any new purchase or donation. 

• Current software titles for productivity tools, integrated software, desktop publishing and graphics 
should be maintained. 

• Authoring tools that parallel the needs of the region should be maintained. Multi-platform 
authoring tools are preferred. 

• Laboratory security should be maintained on all stations through lock-downs, student passwords, 
and limited access to files.” (Northrup & Little, 1986, p 217) 

Preservice Teacher Preparation 

Data from the ISTE Guidelines, Florida’s Preprofessional Accomplished Practices for Technology, and 
results from the Key Informant Meeting were clustered by category. The national and state guidelines suggested the 
following categories: (a) computer basics; (b) connecting peripheral devices; (c) productivity for administrative use; 



(d) productivity for classroom use; (e) integration of technology; (f) multimedia authoring and presentation tools; (g) 
telecommunications tools; (h) instructional design; (i) software selection and evaluation; (j) copyright, ethics, and 
equity; and (k) locating journals and more information on instructional technology. In addition, the Technology 
Teacher Preparation survey disseminated regionally to 300 K-12 teachers, indicated that word processing was the 
most needed skill to be taught in preservice instructional technology courses with 72% of the teachers surveyed 
responding that above average to extensive preparation must occur prior to entering the classroom. Other skills 
reported by practicing teachers include: productivity tools, multimedia tools, predeveloped software; and the Internet. 
The most critically rated productivity tools include: graphics programs (48%); desktop publishing (46%); 

spreadsheets (41%); and databases (40%). Multimedia tools include CD-ROM (46%) and laserdisc (45%). 
Predeveloped instructional software preparation includes drill and practice (42%); problem solving software (46%); 
tutorials (40%); and simulations (37%). The following benchmarks for teacher preparation reflect the knowledge, 
skills, and abilities of a graduate of a teacher preparation program. See Table 1 for a complete description of 
suggested technological skills that preservice teachers should attain as reported by veteran teachers. 

Table 1. Teacher Perception of Instructional Technology Preparation in Preservice Teacher Education Programs 



Technology 


Above Avg 


Extensive 


Total Prep 




Prep 


Prep 




Word Processing 


37 


34.6 


71.6 


Graphics 


32.1 


16 


48.1 


Problem Solving 


25.9 


19.8 


45.7 


CD-ROM 


29.6 


16 


45.6 


Desktop Publishing 


25.9 


19 


44.9 


Laserdisc 


34.6 


9.9 


44.5 


Drill & Practice 


25.9 


16 


41.9 


Tutorials 


23.5 


17.3 


40.8 


Spreadsheets 


27.2 


13.6 


40.8 


Database 


28.4 


11.1 


39.5 


Simulations 


22.2 


14.8 


37 


Research/Intemet 


25.9 


11.1 


37 


TV/classroom use 


21 


13.6 


34.6 


Authoring 


28.4 


6.2 


34.6 


E-mail 


22.2 


11.1 


33.3 


Downloading Files 


21.3 


10 


31.3 


Uploading Files 


21 


9.9 


30.9 


Scanning 


21 


2.5 


23.5 


Using newsgroups 


18.5 


4.9 


23.4 


Photo CD 


19.8 


2.5 


22.3 


Digital Video 


17.3 


3.7 


21 


Network Setup 


12.3 


8.6 


20.9 


Digital Camera 


17.3 


2.5 


19.8 


Network Maintenance 


13.6 


4.9 


18.5 


Audio CD 


16 


2.5 


18.5 



Note: Perceentage of Above Average and Extensive Preparation included. 
N=87 



Benchmark: Preservice Teacher Preparation 

• “Operate a microcomputer system to include powering-up the computer, installing programs, 

accessing programs in other drives (such as CD-ROM in drive d :/), saving files to diskette, and 
deleting files within the context of a Macintosh, Windows, and DOS-based system. (* *) 
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• Set up and operate peripherals such as scanners, laserdisc players, CD-ROM, modems, and 
printers. (*) 

• Use productivity software to include word processors, spreadsheets, databases, desktop publishing 
and graphics to perform basic administrative tasks such as mail merge, grading, and creating basic 
newsletters. (*) 

• Integrate productivity software including word processors, spreadsheets, databases, desktop 
publishing and graphics into a classroom curriculum assignment. Examples may include using 
databases to gather and store information on hurricanes, spreadsheets to record daily temperatures 
across the country and predict weather patterns, and desktop publishing/graphics to publish a paper 
on the temperature highs and lows across the southeast. 

• Use presentation software to present group integration projects. (*) 

° Set up a telecommunication station by installing communication software and by configuring the 

modem to appropriate settings. (*) 

o Use telecommunication to send and receive e-mail and to access network resources through World 

Wide Web, gopher, telnet, or ftp. (*) 

• Integrate telecommunication including electronic communication and information access into a 
classroom curriculum assignment. Examples may include following the migration of whales on 
WhaleNet, tracking the shuttle mission on NASA Spacelink, or tracking the dog sled races to the 
North Pole. 

• Use multimedia authoring tools to create instructional lessons. (*) 

• Use multimedia CD-ROM, laserdisc, and digital photography to access and store information. (*) 

® Integrate multimedia including authoring tools, CD-ROM, laserdisc, and digital photography into 

a classroom curriculum assignment. Examples may include creating a multimedia-authored lesson 
on the migration of whales using laserdisc, CD-ROM, and digital photography. 

• Apply principles of instructional design to evaluate and select software appropriate to a given grade 
level, subject, or varying exceptionality. 

• Promote ethical and legal behavior in the use of instructional technology for school and home 
use.” (Northrup & Little, 1996, p 218) 

Graduate Level Preparation 

Data for this section were gathered from the ISTE Guidelines for Graduate Programs in Instructional 
Technology and recent research in the field. Students interested in receiving graduate degrees in instructional 
technology must be trained as decision-makers in the field and must gain advanced knowledge, skill, and ability in 
the design, development, and application of emerging technologies for teaching and learning (Northrup & Little, 
1996). The following are benchmarks established for graduate level instructional technology preparation. 

Benchmark: Graduate-level Preparation 

• ‘‘Meet all basic hardware knowledge and use requirements specified in the undergraduate benchmarks 
(denoted by an *). 

• Apply technology tools to solve a variety of problems for teaching and learning with technology. 

• Make decisions regarding the use of specific technologies based on empirical research on learning 
effectiveness. 

• Create a technology plan based on a defined critical need in search of a resolution. 

• Synthesize available research on instructional design and current theories of learning to create 
instructional courseware using a selected authoring tool. 

• Synthesize available research on instructional design and current theories of learning to create 
multimedia and hypermedia products using authoring tools, video clips, sound, animation, and 
hypertext links. 

• Synthesize available research on instructional design and current theories of learning to create 
telecommunication simulations and active environments for learning by using available 
telecommunication resources on the Internet. 
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• Synthesize available research on instructional design and current theories of learning to design and 
deliver instruction using distance education. 

• Analyze critically the ethics and equity of technology in K-12, higher education, industry, and for 
home use. 

• Develop a comprehensive plan for managing a large-scale instructional technology system. 

• Conduct research on the effectiveness of instructional technology on specified variables of interest 
for learning, motivation, and/or learning strategies to further the field of instructional technology.” 
(Northrup & Little, 1996, p. 219) 

Teacher Educator Models 

According to Brooks & Kopp (1989), building strong faculty models of instructional technology utilization 
in the classroom is critical to the success of entire programs. A review of the literature has revealed that most 
faculty who do not integrate technology into their teaching practice report lack of access, lack of time, and lack of 
support as the most critical reasons. Additional data sources for the establishment of benchmarks in the area of 
faculty modeling includes NCATE standards, specifically that faculty must model the integration of computers and 
technology in their fields of specialization. Specific benchmarks were established for this area based on NCATE and 
a review of instructional technology standards required by K-12 students through Florida’s Sunshine State Standards, 
as our faculty are encouraged to model and teach based on Florida’s Sunshine State Standards. The following 
benchmarks are designed as a model for training faculty to integrate instructional technology into their classroom 
teaching experiences: 

Benchmark: Teacher Educator Models 



• “Master basic instructional technology skills in the area of productivity, multimedia, 
telecommunication, and classroom integration. 

• Use presentation and multimedia software as a medium for the delivery of class lectures. 

• Use presentation and multimedia software to deliver speeches, lectures, and seminars at 
professional conferences. 

• Require students to word process all final work for course assignments. 

• Require students to work collaboratively to make group presentations using technology. 

• Model and require students to use technology for problem-solving tasks using telecommunication, 
CD-ROM, and laserdiscs to access and apply information, databases for information storage and 
retrieval, and spreadsheets for prediction and hypothesis testing. 

• Use computer-assisted instruction as an alternative instructional delivery medium. 

• Use teacher utility tools for grading, recordkeeping, and test generation. 

• Introduce new software in content-related courses relevant to teaching and learning specific topic 

areas such as Language Arts, Mathematics, etc.” (Northrup & Little, 1986, p 220) 

Inservice 

SERVE (1993) reports that inservice is the most important approach for teachers to become technologically 
literate and to receive current information. However, many current inservice initiatives are not outcome-driven and 
are not expected to produce measurable results after training has been received. To establish benchmarks for inservice 
and outreach, data were clustered in two ways: (1) by topic area for inservice and (2) the responsibilities of the 
higher education institution in providing inservice to veteran teachers. The first cluster, topic area, was defined by 
the aggregation of the ISTE guidelines, Accomplished Practices for Instructional Technology, and Florida’s Sunshine 
State Standards. The results parallel the findings of the preservice teacher preparation benchmarks. The teacher 
preparation benchmarks have actually been adopted by school districts as their plan for inservice activity. 

The second cluster, how higher education institutions can facilitate inservice activities is a result of data 
gathered through key informant surveys. The results are as follows: 

Benchmark: Inservice and Outreach 

• “Provide several opportunities per year for teachers to attend short courses and workshops using 
instructional technology faculty (after school, weekends, and during the summer). 
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Provide a mechanism for ongoing support after teachers attend workshops, courses, etc. 

Promote at least one event annually bringing teachers, administrators, and teacher preparation 
faculty together to learn more about instructional technology. 

Collaborate with area school districts on the direction and use of instructional technology through 
Technology Planning. 

Visit area schools to maintain visibility and to learn new ways technology is being used in the 
classroom. 

Serve on school- and district-level instructional technology focused committees. 

Collaborate with industry, hardware vendors, and software vendors to maintain current knowledge of 
regional and national developments in the field. 

Create an informational newsletter and/or an Internet home page for practicing teachers to keep 
abreast of teacher education technology events.” (Northrup & Little, 1986, p 221) 



Conclusion 

Effecting change in preservice teacher preparation and in K-12 education requires a set of clearly defined 
systemic benchmarks as provided through this study. Implementation of this systemic approach in teacher 
preparation programs will prepare preservice teachers for classroom teaching in the 21st Century. Used in K-12 
schools, this approach will assist in developing veteran teachers knowledge, skill, and ability in instructional 
technology while looking at the entire system of adequate laboratory space, continued inservice, and modeling of 
instructional technology for peers and for teachers. The implications from this study are primarily that instructional 
technology change will not occur unless key elements of the system are addressed. The benchmarks specified in this 
study can serve as a guide to teacher preparation programs and K-12 school districts wishing to facilitate this change. 
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Abstract 

The purpose of this discussion is to illustrate and explain the relationship between drawing ability and 
spatial/visual perceptual ability and their potential implications for education and training. This work defines 
drawing , spatial , and visual-perceptual abilities , and their connection to information processing and intelligence. It 
also discusses results of recent studies which indicate art behavior and spatial ability are normally distributed. The 
suggested implication is that all students have drawing and spatial potential that may be developed through art 
education in general and drawing in particular. 

Background 

Drawing may be the most important variable in art performance and ability. “...[I]t is the easiest mode of 
visual arts expression to assign, administer, and measure; more importantly, it has been recognized as basic to 
expression in all art forms and as a correlate of many other attributes, including general intelligence...”) Stalker as 
cited in Clark, 1989, p. 99). In fact, some believe the development of this skill is important to the total 
development of the child. As Robach states “...I have come to believe that drawing is the missing link to American 
success in educating every child” (1994, p. 3). 

Given the above assertion, why doesn’t the education system place more emphasis on the development of 
visual-perceptual skills? This is not a foreign concept. It has long been a consensus among art educators “...that 
developing or increasing visual readiness is one of the major goals of a good art program” (Herberholz & Alexander, 
1985, p. 22). A brief look at the history of art education may shed some light on this dilemma. 

The form and function of art education has changed over the years due to the change in the philosophical 
underpinnings. Art in the schools had various objectives from the practical needs of life (crafts oriented) and the 
development of hand-eye coordination and the acquisition of mechanical skills necessary for the industrial age 
(Eisner, 1972). In the latter half of the 20th century the concept of innate creativity came to the fore changing the 
focus of art education from specific skills training to that of enrichment of the total individual. As a result, the role 
of the art teacher changed from instructor to nurturer. Instead of the rigid training of specific skills, the teacher 
became a provider of materials who encouraged self-expression.. The art teacher’s function became that of observer 
and facilitator thus reflecting one of today’s popular philosophies in art education (Gardner, 1990). 

This approach to education in the visual arts has had a notable impact on the perception of the learner both 
from the teachers’ and students’ perspectives. Students who had more developed visual perceptual skills succeeded in 
meeting or exceeding expected standards for art production. This led to the belief that only some individuals were 
“talented” or had “special gifts”. Accordingly, some educators, students, and the general public have come to believe 
that art skills are a behavioral peculiarity attributed to only a few (Clark & Wilson, 1991). 

Recent studies, however, suggest art behavior is normally distributed (Clark, 1993; Edwards, 1979; Gardner, 
1982; Gardner, 1993). Others also believe there is a link between art and spatial ability (Eliot & Smith, 1983). 
Visual perception is needed for the development of drawing and spatial skills as the sensory input for both is 
primarily visual. If visual-perception can be taught and learned through drawing training, then the concept of artistic 
“giftedness” may be redefined as a skill potential that may be developed beyond an expected norm resulting from life 
experience alone. Consequently, if spatial ability is related to, and possibly improved by drawing, i.e., visual- 
perceptual training, perhaps it is feasible to train perception through drawing. 

Drawing Ability 

Drawing exhibits principles of design some of which can be evaluated by a test instrument called the 
Clark’s Drawing Abilities Test (CD AT) (Clark, 1987). These drawings may be either representational or stylized 
(see Figure 1) without effecting the scoring rubric. Each task on the CDAT is rated on a scale of one to five. 





Figure 1. Realistic (right) and stylized (left) interpretations of a runner as drawn by two eighth graders. CD AT 
drawing task number 2. Both scored 5. 



Both representational and stylized drawings may score 5. Concepts examined in each of the tasks are listed below: 



Task 1 


Task 2 


Task 3 


Task 4 


perspective 


action 


receding space 


imagination 


texture 


body proportion 


grouping 




size 


recognition of detail 







recognition of detail 



Spatial Ability/Visual-Perceptual Ability 

Spatial ability has been variously defined over the years with researchers debating over the exact function 
and number of contributing factors (Cohen, 1982; Dixon, 1983; Egan, 1979; Eliot & Smith, 1983; Gardner, 1993; 
Horn & Smith, 1945; McGee, 1979; Smith, 1964; Pellegrino & Kail, 1982; Pellegrino & Hunt, 1991; 
Youngblood, 1979). John Eliot (1983) combined various definitions into a general statement that spatial factors 
reflect "... the perception and retention of visual forms and the mental manipulation and reconstruction of visual 
shapes” (p. 9). Linn & Peterson (1985) defined spatial ability as representing, transforming, generating, and 
recalling non-linguistic information. All of these mental operations are necessary to drawing and shall be discussed 
in a subsequent section. 

Although many factors of spatial ability have been identified, researchers most often recognize three: spatial 
relations, spatial orientation, and spatial visualization (Rowe, 1991). These definitions coincidentally correspond 
almost verbatim to those used in the Test of Visual-Perceptual Skills (non-motor) (TVPS) which is used solely to 
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evaluate mental capabilities related to spatial ability. TVPS terminology is also in agreement with other researchers’ 
definitions of the spatial ability factors (Cohen, 1982; Dixon, 1983; Eliot & Smith, 1983; Lacey & Guilford as 
cited in Eliot & Smith, 1983; Lohman as cited in Pellegrino & Kail, 1991; J. W. French as cited in Eliot & Smith, 
1983; Rowe, 1991). By definition, spatial and visual-perceptual abilities are synonymous. Gardner’s definitions used 
in the TVPS are as follows: 

Visual discrimination f Spatial relations! “A child’s ability to remember for immediate recall (after four or 
five seconds) all of the characteristics of a given form, and being able to find this form from an array of similar 
forms.” 

[Sp ati al Qri gn ta tionL “A child’s ability to determine, from among five forms of 
identical configuration, the one single form or part of a single form that is going in a different direction from the 
other forms.” 

Visual form constancy [Spatial visualization!, “A child’s ability to see a form, and being able to find that 
form, even though the form may be smaller, larger, rotated, reversed, and/or hidden” (Gardner, 1988, p. 65). 

Drawing and Spatial Abilities 

The CDAT focuses on a number of standard elements of art as criteria in its analysis of drawing 
performance. By comparing these against the definitions for spatial/visual criteria, a theoretical common ground 
may be identified. 

Spatial/Visual Criteria Drawing Criteria 



• Relations — recall and find characteristics 


• perspective 


• texture 


of a given form 


• proportion 


• size 


• Orientation — identify form or part going 


• shape 


• detail 


in a different direction 


• action 




• Visualization — identify a form changed in size, 


• receding space 


• grouping 


position, direction or hidden 


• imagination 





Paul Messaris cited Howard Gardner’s (1983) question: “...because spatial intelligence contributes to 

picture-making ability, might spatial intelligence be an area of cognitive functioning enhanced by experience with 
images?” (Messaris, 1994, p. 27). In fact, several specific subject areas have been identified as correlating with these 
abilities. Eliot & Smith pointed out several positive correlations between spatial ability and a general art class 
(.404) and specific art-related skills such as technical drawing (.45), metal work (.485), mechanical science (.492), 
and woodworking (.67) (Eliot & Smith, 1983, p. 436 & 442). 

Relationships to Intelligence 

Educational achievement and artistic performance — including drawing — require proficient spatial abilities. 
The importance of spatial ability in learning, communication, and life skills has become increasingly evident. 
Spatial ability positively correlates with success in areas such as mathematics, physics, technical drawing and 
woodwork (Eliot & Smith, 1983). Taking this a step further, Smith asserted that “...spatial ability has a wider 
significance than as mere aptitude for courses in art and technical subjects” (Smith, 1964, p. 100). Many other 
areas — engineering and interior decorating to name a few — require analytical visual skills and already have a proven 
relationship with spatial abilities (Eliot & Smith, 1983). A relationship between spatial ability and certain types of 
intellectual adeptness has also been well established (Biederman, 1948; Caldwell & Moore, 1991; Cohen, 1982; 
Egan, 1979; Eisner, 1972; Gibson, 1953; Gardner, 1982; Gardner, 1993; Goins, 1958; Goodenough, 1926; Kolers, 
1979: Laurendeau & Pinard, 1979; Loyacono, 1993; Messaris, 1994; Olson & Bialystok, 1983; Pellegrino & Kail, 
1982; Pellegrino & Hunt, 1991; Salomon & Perkins, 1987; Sherman, 1947; Smith, 1964; Szeto, 1975; West & 
Morris, 1985). Academic, career, schooling, and visual decisions along with visual and verbal communication are 
just some of the everyday skills which require space, distance, and directional capabilities all of which depend on or 
are related to spatial abilities. 

Evidence suggests a relationship between inherent art ability and general intelligence (Clark, 1989; Clark, 
1993; Clark & Wilson, 1991; Clark, Zimmerman & Zurmuehlen, 1987; Cook, 1985; Cunningham & Reagan, 



1972; Eisner, 1972; Gardner, 1982; McFee, 1970; Messaris, 1994; Shannon, 1991; Tiebout & Meier, 1936). 
Messaris supported the concept of this relationship when he stated, “It is conceivable that the enhancement of depth- 
perception abilities [spatial] might lead to a more general stimulation of one’s capacity for perceiving and thinking 
about three-dimensional space, and in that case we would be dealing with a type of cognitive activity that plays a 
very important role in general intelligence’’ (Messaris, 1994, p. 12). 

“Proponents of visual education often argue that experience with visual media is not just a route to better 
visual comprehension but also may lead to general enhancement of cognitive abilities” (Messaris, 1994, p. 3). 
Drawing, through its exercise of cognitive functions common to many modes of expression, may be one way to 
effectively train spatial/perceptual abilities, promote visual literacy, and enhance intellectual performance. 

This perspective gives rise to questions concerning the relationships between drawing ability, spatial 
ability, and intelligence and/or cognitive skills (see Figure 2). 



Intelligence 




Figure 2. Conceptual map of interrelationships and suggesting the questionable link between drawing ability and 
spatial ability. 

Some studies show that students with greater spatial/perceptual skills are more skilled in visual 
interpretation of their reality (drawing ?) and achieve greater success in areas such as math, science, and others 
requiring visual perceptual skills. Spatial test results reported by Eliot & Smith (1983) suggest there is a significant 
relationship between scores in these academic areas and spatial ability. The relationship between spatial ability and 
certain academic and skill areas leads to the question of how to delineate and train spatial ability for the purpose of 
developing cognitive skills. “There is clearly a very great need to devise methods for identifying those different types 
of ability which are necessary for success in subjects such as mathematics and science” (Smith, 1964, p. 23). 
Assuming there is a relationship between drawing and spatial competencies, drawing may be one of these abilities 
that are necessary to academic success in general and visual-perceptual dependent areas specifically. 

Normal Distribution 

The debate over whether or not the ability to draw and spatial ability are gifts of nature or products of 
nurture continues (Eliot & Smith, 1983; Gardner, 1982; Gardner, 1990; Lewis, 1973; McFee, 1970; McGee, 1979; 
Olson & Bialystok, 1983; Smith, 1964). Perhaps finding a common ground will help clarify at least some of the 
surrounding issues. One of these areas of commonality is trainability. Both drawing and spatial skills can be 
trained. There is evidence to support the trainability of spatial competence (Cunningham & Reagan, 1972; 
Sherman, 1947; Szeto, 1975). In 1919 Manuel made the same suggestion regarding artistic ability when he 
questioned, “Interest [in art-related endeavors] may indicate either a superior innate ability or merely a rather high 
development of a rather ordinary endowment...” (as cited in Clark, Zimmerman, & Zurmeuhlen, 1987, p. 15). These 
two observations seem to suggest common potentials developed through uncommon effort. 

Since drawing and spatial/perceptual skills can be trained, this suggests both are normally distributed 
potentials such as reading and math, and therefore, can benefit from instruction (Edwards, 1979). Nature then plays a 
limited role by supplying a core of potential aptitudes, while a nurturing environment may stimulate and develop 
them. Since it has already been established that visual-perceptual skills can be trained and that they are synonymous 
with spatial skills by definition, it follows that each would benefit from the same training. Learning to draw may be 
one of these common areas. Referring back to their respective connections to intelligence, one might deduce that 
nurturing visual-perceptual skills through drawing training would enrich all three areas (see Figure 2). 



Information Processing/Training 



Visual perception is learned (McFee, 1970). Drawing and spatial or visual-perceptual skills have similar 
brain functions for the processing of information. Drawing, as an output of visual perception, enables the 

conversion of an abstract visualization to a concrete product. It involves a cognitive operation, sometimes called 
information processing, which allows transference of one type of information to another (Biederman, 1948; Gardner, 
1982; Pellegrino & Hunt, 1991; Sherman, 1947). Thus, an abstract mental picture (a visual perception) can be 
translated into a concrete drawing. The more developed the spatial/perceptual skills, the more sophisticated the 
attending representation. 

These mental operations also benefit from similar training methods, i.e. tactile and kinaesthetic. Visual and 
tactile sensual stimulations are the primary sensory inputs needed for drawing and one mode of training spatial 
ability (Meumann as cited in Clark, 1989, p. 99). “The development of perceptual skills is an often stated goal of 
art education. Both art educators and psychologists have referred to the relationship between perception and child art” 
(Grossman, 1970, p. 51). This relationship between perception and art manifests itself more specifically in the 
ability to draw. “Visual perception, as the term is used here, is the process by which phenomena are apprehended by 
the mind through the medium of the eye” (Goins, 1958, p. 1). As a necessary skill in mental development As 
evidenced by the known link between general intelligence and drawing ability, perceptual skill development is also 
recognized as a necessary skill in mental development (Clark, 1989; Clark, 1993; Clark & Wilson, 1991; Clark, 
Zimmerman & Zurmuehlen, 1987; Cook, 1985; Cunningham & Reagan, 1972; Eisner, 1972; Gardner, 1982; 
McFee, 1970; Messaris, 1994; Shannon, 1991; Tiebout & Meier, 1936). It appears spatial development parallels 
artistic development and is dependent on tactile and kinaesthetic motor activity. 

For it is not until after 7-8 years of age that measurement, conceptual co-ordination of 
perspective, understanding of proportions, etc., result in the construction of a conceptual space 
marking a real advance on perceptual space. All the same, it is worth noting that despite their 
differences and the time lag which separates them, both perceptual and representation 
construction are to some extent repetitive and possess a factor in common. This common 
factor is motor activity. (Piaget & Inhelder, 1967, p. 13). 

Sherman (1947) rigidly applied motor training techniques using drawing for training of visual perception. 
He noted that the increase in visual perceptual skills after drawing training resulted in improved visual acuity, for 
example, clarity of central vision and the ability to identify a shape of a certain size at a certain distance. The 
symbiotic nature of these functions was further extolled when Cunningham & Reagan (1972) pointed out that 
cognitive abilities and emotional development were also enhanced by such training. 



Implications 

Clarification of the interrelatedness of the above elements and the fact that they are independently related to 
general intelligence has important implications for art education and curriculum development. As an instructional 
strategy and training, drawing may potentially be used to promote cognitive skills as well as those related more 
directly to visual perception. The connection to visual literacy, the ability to read and write visually, is central to 
this discussion. Drawing is a form of visual communication and, as such, an implement of visual literacy which 
allows one to communicate effectively graphically. Visual perceptual and spatial skills enable visual 
communication resulting in visual literacy. By acknowledging the relationship between drawing and spatial ability, 
we may more accurately design art education curricula to teach visual literacy and meet these learning needs of 
children. Instructional design concerns relative to these outcomes should not only apply the appropriate strategies 
and materials for development of these skills, but also take into consideration the sequential ordering, presenting, and 
training of spatial and visual-perceptual skills for the development of cognition and for more efficient information 
processing in general. 

Despite the fact that there will always be individual differences, knowledge of visual-perceptual and drawing 
processes can have important implications on instructional design for technology application and integration. 
Computers and multimedia in particular are an ever-present and growing force in today’s society. They are visual 
media requiring skills in visual communication as well as those of verbal language. Training in visual 
literacy/communication through visual-perceptual and drawing training may enhance the effectiveness of the use of 
this medium whether it be as developer or user. Computers are the key piece of hardware in instructional 
technology, and a powerful influence in today’s education. Students and educators must be able to use both verbal 
and visual languages effectively. “Recently, educators have begun to realize that, first, this visual age requires visual 
as well as verbal skill of everyone, and second, the verbal and visual skills are interconnected and both must be 
developed” (Fransecky & Debes, 1972, p. 9). Drawing training, because of its relationship with visual literacy and 





other cognitive areas, is essential to the total educational development of the child and has particular significance 
with respect to the ever- advancing communication technologies. 

Summary & Conclusion 

Drawing and spatial abilities share common conceptual ground, training and outcomes: 

• Drawing and spatial abilities share common conceptual ground. Both have been associated with 
similar cognitive functions. They are related in mental function through a relationship in which 
drawing is dependent on spatial ability and spatial ability may be enhanced through drawing. 

• Drawing and spatial abilities share common training. Both mature through tactile and kinaesthetic 
experience. Drawing is learned through repeated practice of a tactile and kinaesthetic nature. Spatial 
ability can be developed by physical manipulation and movement. 

• Drawing and spatial abilities share common outcomes. Skills in each area directly relate to success in 
similar academic and professional fields which require visual-perceptual accuracy. 

Art is not a frill, not does it train only fine motor skill. It also helps develop higher order thinking skills 
(Feldhous, 1992). Drawing is a means of planning and assembling ideas (Caldwell, & Moore, 1991). Another 
perspective looks at drawing as communication which represents unity in our minds (VanSommers, 1984). All 
expound the benefits of drawing. 

Art is not an end, but a means to express, think, explore, satisfy, and solve. It is an entire group of media 
which, when appropriately applied, lead to learning, expression, visual literacy, and satisfaction. The language of 
drawing, as a single art form, crosses media boundaries and enables visual communication as well as heightening 
cognitive capabilities. By nurturing visual-perceptual skills through drawing training, spatial abilities may be 
enriched thus enabling transference of intellectual skills. I. Macfarlane Smith (1964) feels “...spatial ability has a 
wider significance than as a mere aptitude for courses in art and technical subjects”. He goes on to quote Myers’ 
1958 study: 

In our judgment, spatial ability is an important and pervasive trait, affecting our perception of our 
environment and our style of thinking about it. When better tests are built and a better theory 
provided for their use, we believe it possible that we will find spatial ability to be similar in 
importance to such traits as verbal or social intelligence, (as cited in Smith, 1964, p. 100) 

Educators in general and art educators specifically, may consider these concepts when analyzing their 
instructional goals. Subsequent research of an experimental nature may verify the connection between drawing and 
perceptual skills and intelligence with the possibility of predictability and the goal of educational reform. More 
research is needed on the training of spatial ability and its transferability to academic tasks. 
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Abstract 

How can researchers establish the trustworthiness of data collected using electronic mail? This study 
illuminated some of the substantive issues related to this methodology and provided researchers with twelve 
guidelines for using this medium. Results suggest that researchers can cautiously rely on electronic interviews and 
that compared to the traditional approach, there are advantages , such as streamlined data collection, transcription, and 
data analysis. 

Purpose 

How can researchers establish the trustworthiness of data collected using electronic mail? This study 
attempted to illuminate some of the substantive issues related to this methodology and to provide researchers with 
protocols, suggestions, and guidelines for using this medium. The advantages and disadvantages, as identified in this 
study, are outlined. 

Theoretical framework 

Lincoln and Guba (1985) make a distinction between qualitative methods and qualitative inquiry. While 
interviewing is generally viewed as a qualitative method, use of interviewing does not necessarily lead to qualitative 
research. Indeed, the qualitative paradigm requires, according to Lincoln and Guba, that: (a) the researcher carry out 
the inquiry in ways that are consistent with the entirety of qualitative beliefs, (b) the researcher relies on human 
instrumentation, (c) the researcher carefully plans the nature and strategies of the study, and (d) the researcher 
becomes familiar with the site in which the study will take place. Certainly a study in which the researcher 
conducts interviews using electronic mail may meet these criteria. It is also likely that the interviewing may take 
place as data collection in a more conventional, quantitative study. Thus, interviewing in the context of this study is 
not meant to be seen as qualitative or quantitative but rather as a method of data collection. 

Interviews, according to LeCompte and Preissle (1993), elicit information from participants through 
personal interaction and "they share an advantage over less obtrusive measures like questionnaires because researchers 
guide the revelation of information" (p. 165). A strength of interviewing as a valid method of data collection is that 
information can be checked thoroughly and repeatedly for accuracy. "The design of qualitative interviewing provides 
the tools for encouraging people not to lie, for detecting bias, and for compensating for bias when it does occur" 
(Rubin & Rubin, 1995, p. 225). 

Researchers using electronic mail as a method of data collection should be concerned about a variety of 
potential problems including difficulties related to establishing rapport with respondents along with the 
encouragement and interpretation of emotion or meaning based on nonverbal cues. Establishing rapport and 
encouraging nonverbal cues may be difficult in traditional face-to-face interviews. It is likely that such interview 
relationships may be more easily established with electronic mail users since these characteristics are often reflected 
by users of electronic mail. According to Rubin and Rubin (1995), interviewers need to address the special symbols 
and terminology used by the interviewees in order to gain insights into their understandings. Electronic mail users 
imbed unusual symbols and acronyms within their text and investigators should become familiar with them as a 
useful tool for developing electronic interview relationships. 

In settings similar to electronic mail, researchers have found that self-disclosure is more likely when 
subjects are anonymous. For example, in decision centers, where groups of participants respond using computer 



terminals and are not identified by name, coordinators report that dialog is progressive, meaningful, purposeful, and 
highly stimulating (Aiken, 1993; McGrath, 1986). Subjects/participants using electronic media have the 
opportunity to be thoughtful, reflective, and honest in response to questions and comments from a facilitator (Rubin 
& Rubin, 1995). The electronic mail medium offers a greater degree of anonymity for most users than other types 
of media. 

Electronic mail has been used recently in research for a variety of supplemental and tangential activities 
within projects of a larger scope, as well as a primary data source. For example, Workman (1995), in a participant 
observation study of a computer systems firm, scheduled appointments, kept track of meetings, corresponded with 
key informants to clarify issues via electronic mail, received many documents including agendas and minutes, 
followed activities of certain groups by reading electronic bulletin boards and participated in special interest group 
discussions on-line. Foster (1994) used electronic mail to interview new subscribers to a listserve. He was 
interested in studying the ways in which universities conduct curriculum planning. Foster concluded that there were 
both advantages and disadvantages, as well as some potential problems, associated with electronic data gathering. 
Contributions of the electronic medium included: (a) electronic mail could provide a supplemental data source; (b) 
scheduling and geographic location problems are eliminated; and (c) cost, time, convenience, and interview form 
could have huge advantages over traditional interviews. However, electronic interviewers should be especially 
cautious in regard to ethics and common courtesy. In addition, Foster raises the issues of sampling bias and 
generalizability when using electronic media. 

Sampling bias may be an extremely important issue for those conducting research using electronic mail. 
Anderson, Bikson, Law, and Mitchell (1995) studied the composition of household and workplace computer 
ownership and use. These researchers found the majority of owners and users were white, educated, and middle-class. 
They also found small differences in age and gender. Thus, researchers using electronic mail to gather data may be 
“contributing to information apartheid” (M. Ender, personal communication, March 4,1996). However, it is also 
possible that subjects previously inaccessible to researchers may become available using electronic media. In this 
way, sampling opportunity may compensate to some degree for sampling bias. 

Method 

Study participants were selected using the following criteria: participants worked in an environment in 
which they were at ease using electronic mail for communication, they were geographically dispersed, and they 
regularly relied on electronic mail for communication. Possible participants were identified by networking with the 
researchers’ colleagues. Twelve subjects were selected and invited to participate in the study (see Appendix A). 

At the onset of the study, the interviewers attempted to establish a rapport by engaging in a series of 
informal discussions; in this way, the interviewers became familiar with the respondents’ individual communication 
styles and each dyad began the slow process of becoming acquainted. Six of the original respondents dropped out of 
the study for various reasons. The six remaining respondents continued to have a fairly consistent dialog with their 
interviewers over a period of about four months. 

The researchers’ intent was to investigate the use of electronic interviewing as a viable method of data 
collection. The substantive nature of the interviews was not of particular interest except as a means to assess the 
method. The interviews were characterized as being informal and conversational (Patton, 1990). Interviewees were 
encouraged to use typical expressions of emotion, such as : ) (smiling), all caps (yelling), or acronyms such as LOL 
(laughing out loud) and to use other, even innovative, ways of expressing their feelings or emphases. (These 
expressions are often referred to as “emotext” or “emoticons.”) Semi-structured, open-ended questions were used 
initially and followed by clarifying probes and prompts. The substantive topic addressed in the interviews was 
related to the respondents’ use of technology and included the following types of probes: how participants currently 
use technology, how their use has changed over the past few years, if they had access to a “technology mentor,” and 
what their hopes and fears were for future technology use. However, as expected, the direction of each interview 
differed among the participants based on their individual responses to the questions. The number of communications 
differed for each dyad. The interviewers carefully documented difficulties and accomplishments throughout the 
interview process, comparing and contrasting electronic interviewing with traditional face-to-face interview protocols. 
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Results 

Many of the difficulties inherent in face-to-face interviews were overcome in the electronic medium. The 
researchers were unable to interrupt the interviewees or to give nonverbal evaluative responses. Both the researchers 
and the interviewees were able to take the time to be thoughtful and careful in their responses to each other, 
increasing the depth of understanding for both parties. Neither the researchers or the interviewees had to schedule 
appointments or be concerned with the effects of interruptions. Data recording was simplified in this study, 
compared with traditional interviews. In face-to-face interviews, researchers must decide whether to make hand- 
written notes or use a videotape or audiotape; each method has comparative advantages and disadvantages (Rubin & 
Rubin, 1995). In this study, the interview and the recording were simultaneous. As a result, the process involved in 
data analysis was streamlined; the usual transcription process was completely circumvented. Responses, therefore, 
were accurately and thoroughly collected. 

The user characteristics of the sample both enhanced the trustworthiness of the study and led the researchers 
to question the credibility of electronic methodology. We were able to enlist study participants without regard to 
geographical location. However, the sample was limited to those who had access to a computer and to an on-line 
service. So, in one regard, we were able to draw our sample from a more diverse geographical population, but in 
another, our sample was biased toward the “information rich while ignoring the information poor” (M. Ender, 
personal communication, March 4,1996). 

Unexpected difficulties with the method surfaced early in the study. We experienced several “false starts,” 
learning that it was very important to establish a rapport prior to interviewing. This factor may be much more 
important in electronic interviews than in face-to-face interviews because the interviewer does not have as much 
opportunity to interpret nonverbal cues. Additionally, we lost contact with some of our original participants for a 
variety of reasons. Some participants appeared not to use electronic mail in the summer months; others woe 
inconsistent with their frequency of use; and several apparently disconnected from their service provider. 

Discussion 

Advantages of interviewing using electronic mail rather than traditional face-to-face interviews include 
reduced cost and time, convenience, unimportance of geographic location, possibly more sampling diversity, large 
amounts of data may be accumulated very quickly, follow-up and clarification may be more thorough and thoughtful, 
recording and transcription can be accomplished in a single step without interfering with the actual interview, no 
danger or discomfort for the researcher, and the interview process can continue until the researcher is satisfied that a 
saturation point has been reached (Lincoln & Guba, 1985). 

As a result of our reflective inquiry, we suggest the following guidelines for researchers who are considering 
conducting research via electronic mail using interviews: 

1 . Select the sample carefully using specific criteria and with the understanding that sampling bias 
will likely exist. Possibly the selection criteria should be even more clearly defined than is usual in order to 
compensate for bias. Find out if the possible participants use electronic mail only in one location (at work, for 
example) and, if so, be sure that he or she will have consistent access during the period over which the interview will 
be conducted. 

2. Establish guidelines a priori with the interviewees regarding time between communications, 
whether any other type of communication may be expected (e.g., telephone, regular mail), full disclosure that they 
are participating in a research project and the correlated ethical issues, and other issues related to using interview 
protocols. 

3. Prior to actual interviewing, establish a rapport with the interviewees by “chatting.” This stage of 
the pre-interview process will also help to establish standardized electronic mail response patterns for both the 
interviewer and the interviewee. 

4. Be timely with responses, especially when clarifications, illustrations, explanations, or 
elaborations are needed. 

5. Use acronyms and symbols that communicate feelings, emotions, and the like. Encourage the 
interviewees to do the same. Ask for explanations when new expressions are introduced. 

6. Summarize the interviewee’s responses to previous questions and return the summary to the 
interviewee immediately for verification. This will demonstrate understanding and concern for careful representation 
while allowing for clarification of misinterpretation. 

7. Check for messages from interviewees regularly. 
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8. Limit the length of messages to interviewees. Break questions into small parts and ask only a few 
questions at a time. Electronic mail users characteristically write brief messages but will respond consistently. 

9. Be alert for misunderstandings. Be attentive to changes in the ‘tone’ of responses, unusual 
response lag, symbols that are inconsistent with previous dialog, and any other clues which might lead you to 
question the credibility of a response. 

10. Be prepared to re-focus the discussion on the interview topic(s). The electronic relationships which 
develop over these longer timeframes can become quite comfortable for the interviewee and there may be a tendency 
toward self-disclosure beyond the scope of the interview topic(s). Do not overtly discourage this sharing; rather, 
subtly encourage responses related to the research topic. 

1 1. Encourage the interviewees to forward relevant artifacts such as minutes from meetings, messages 
posted in special interest or listserve discussion groups, and mail messages from friends or colleagues. 

12. Be an ethnographer. Study the culture of electronic mail and be careful not to be offensive. For 
example, writing in all uppercase letters implies “raising the voice.” Subscribe to a listserve, if necessary, and spend 
some time “lurking” (defined as simply reading the postings without getting involved in the discussion) so that “net 
etiquette ” or “netiquette” will not be a problem when you begin your research. Be prepared to communicate these 
cultural standards to interviewees if necessary. 

Summary 

Will researchers increasingly rely on electronic mail interviews as a primary data collection source? The 
answer may well depend upon three important factors. First, researchers themselves must feel at ease with electronic 
mail. Certainly qualitative researchers base the trustworthiness of their findings in part on their ability to become a 
participant in the culture they are studying. Researchers with a more positivistic approach would likely use 
electronic interviews in much the same way as a mailed questionnaire or telephone survey: standardized and 

structured. In either case, electronic interviews may be a viable alternative to face-to-face interviews. The second 
important factor is that the interviewees must also feel at ease using electronic mail. If they are not, it is likely that 
they will drop out of the study or response time will be so long that the communication is devalued. For some 
potential interviewees, it may be necessary to engage in lengthy pre-interview strategies to enhance their level of 
comfort with this electronic medium. Finally, researchers must be able to convince others of the trustworthiness of 
their findings; that they have truly represented the electronic voices of their interviewees while maintaining the 
agreed upon level of anonymity. 

Data collected using electronic interviews can be used confidently but cautiously to represent the views of 
participants in research. In order to understand the emotions, meanings, and emphases of respondents, researchers 
should use techniques and terminology that are particular to users of electronic media. It is possible that this 
research will extend to other electronic data collection methodologies such as electronic focus groups, video 
interviews, Internet chat rooms, and listserves. This research provides a necessary and important link between 
qualitative research methods and technology. 
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Appendix A 

The following is a copy of the message sent to twelve potential participants in this research effort. The same 
message was sent by each researcher involved. 

Potential Participant Name, 

Three researchers at the University of Northern Colorado (Dr. Kay Persichitte, Dr. Suzanne Young, and Major Don 
Tharp, USAF) have identified an area for research in which we hope you will participate. This message is being sent 
to a select group of people whom these researchers have met at AECT, AERA, through professional contact, or 
through personal contact. 

We wish to establish an interview relationship that would allow for, and encourage, open communication between us 
on the general topic of your use of technology. The duration of our interview will depend on the strength of the 
interview relationship that develops. We know that your time commitment in responding to our questions will 
vary. Your responses and our research interests will guide the direction of future queries. Thus, the time 
commitment for each participant in this study will vary. 

If you agree to participate in our study, we encourage you to make frequent and appropriate use of acronyms and 
symbols to best communicate your meaning. For example, if I wanted to show EMPHASIS, I would type in all 
caps. 

We guarantee that you will receive anonymity as a participant in this study. Please do not be concerned that your 
comments will be misused in any way. In fact, you will have the opportunity to read and APPROVE our analyses 
before any dissemination takes place; our primary interest is in understanding YOUR point of view : )! 

We look forward to your reply as to whether you are willing to participate in this interview study. If you are not 
interested, please let us know so that we can expand our participant pool accordingly. 

Sincere thanks for your time, 

Individual Interviewer Name 
Complete Address 
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Abstract 

This paper examines gender messages within television advertisements. It clearly shows the need for 
awareness and education about blatant and subtle messages that carry forth the ideas of gender stereotypes throughout 
our society. The authors raise issues about impact of advertising on personal beliefs ; value systems , and self- 
concept. 



Television advertising is big business. What started out in 1941 as a simple talking head that aired for 
$9.00 has turned into an $850,000 spot for the Super Bowl broadcast (Rutherford, 1994). In the United States, 
commercial advertisers pay stations according to the size of the audience they deliver. Because television stations do 
noteam money from the general public that they serve, they are responsible to the advertisers, not to the public. 
Ploghoft (1982) makes a crude, but accurate, analogy between marketing strategies of television and livestock 
industries. The analogy simply says that the job is to round up the herd by whatever means are necessary, then 
present it to the buyers. The buyers, then inspect the offering to make sure it is of the kind and quality desired. The 
herd of television viewers must meet the needs of the paying advertisers. Like cattle, if the audience is too old, too 
young, or not the right kind, a popular show may be dropped because the advertiser is targeting a different audience. 

The way that advertisers target television audiences is not subtle and due to the great mass of the viewing 
public, the repeated marketing messages greatly influence in our beliefs and buying habits. Thus society is shaped 
by suggestions of television advertisers whose motives are to sell products and who greatly influence our beliefs 
about what we should buy, how we should look, how we should act, and who should do what. A great majority of 
these messages perpetuate stereotypes, either blatantly or subtly. Thus, many advertisements tout the strength and 
confidence of males in terms of work, money, and power, while they target the female audience for glamour, home, 
and child care products. Technology as a means of work or power, like financial matters, targets the male domain in 
terms of both quantity and quality of use. Like the market for video games, the home computing market offers 
promise but so far, it caters to a male audience. Any consideration of the influence of gender stereotyping within 
technology advertising, must examine first the stereotypes within television programming and advertising in general. 

Gender Stereotypes in TV Ads 

Prime time dramatic programming features between two or three males to every female character. Women 
are likely to be younger than men and tend to be cast in limited, stereotypical roles with clerical work as the most 
common job. In the real labor force, women outnumber men as teachers and restaurant workers but in the world of 
television, this is reversed. Thus male roles appear to be important in terms of numbers, leadership, and role 
models. 

Typically, women are depicted as feminine and showing their beauty, grace, style, and sexual attractiveness. 
Yet the imagery of feminism is defined by the television producers and advertisers, not by real women in real life. In 
addition, advertisers like to show women shopping, cleaning, cooking, and looking after others. In the never-ending 
war on dirt, the act of cleaning takes on a meaning of love, order, and conformity. Thus the image of the 
homemaker providing a haven of harmonious safety, regardless of the state of the outside world and its many threats, 
is flashed repeatedly before the eyes of the viewing public (Rutherford, 1994). 

On the other hand, men are associated with the outdoors, sports, cars and driving, relaxing, or entertaining 
at home (Rutherford, 1994). Roles within the work and home environments that command expertise or authority, 
such as recommendations for finances, insurance, making decisions for senior adults, or offering professional 
opinions, typically feature men. In addition, adult males are usually the announcers or authoritative voices for 
products, even for products mainly used by women. 

Most of the experts and voice overs in commercials are male because advertisers operate on the assumption 
that men can sell more products due to their voice of authority. However, research shows that women's voices are 
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just as effective in motivating audiences to buy products (Savan, 1994). For example, after viewing ads for Listerine 
and Nes tie's Morsels in both male or female voice over versions, test audiences chose those products equally, 
regardless of which version they had heard. In fact, they often couldn't recall if the voice over was male or female 
(Savan, 1994). 

The pervasive gender messages that run throughout the programming and advertisements are not limited to 
adult imagery. Children’s programming and advertising follow the typical adult stereotypes. In fact, the very 
structure of the advertising message is skewed, so the audience becomes attuned to recognizing the commercials by a 
quick glance or by hearing them even when they are not watching a full commercial. The male-oriented commercials 
contain more cuts, loud music, and boisterous activity, whereas female-oriented commercials contain more fades and 
dissolves, soft music, and quiet play (Signorielli, 1993). It is typical to see boys playing outside while girls play 
inside, with camera angles pointing up to make the boys look dominant, while they point down and make the girls 
look smaller. 

While gender stereotypes have changed over the years, there is a code that identifies feminine and masculine 
characteristics as portrayed in current television advertising. Rutherford (1994) provides the following examples as 
shown in Table 1. 

Table 1. Gender Stereotypes in Television Advertisements 





Feminine 


Masculine 


Physique 


weak 


strong 


Traits 


emotional 


rational 




delicate 


rugged 




warm 


cool 




sociable 


competitive 




narcissistic 


masterful 


Display 


body 


authority 


Domains 


home 


workplace 


% 


private 


public 



How Many Commercials Do We Watch? 

The National Association of Broadcasting code states that during each hour of prime time and weekend 
children’s programming, the maximum amount of nonprogram material is limited to nine and one half minutes. 
During weekday children's programming, 12 minutes are allowed per hour, and 16 minutes per hour at all other 
times. This averages out to 17 commercials per hour. Children of ages two through five years view an average of 
22,338 commercial advertisements per year and children of ages six through eleven view 18,856 per year. We have 
seen one million commercials by the age of forty (Adler, Lesser, Meringoff, Robertson, Rossiter, & Ward, 1980). 
These figures represent the numbers of commercials within television programming, and might be somewhat higher 
than what the typical person really watches, due to viewing habits. The remote control encourages “channel 
grazing,” so as many as 60 percent of the people might not actively attend to the television during commercial 
breaks, and perhaps only 25 percent of the viewers actually watch commercials (Rutherford, 1994). 

On the other hand, many children whose minds wander during programming, snap back to attention when 
they hear the louder volume of television commercials. About 33 percent graze, zap, or channel surf because they 
are dissatisfied with the channel they are watching, wish to avoid a commercial, or are afraid they are missing 
something better on another channel. The largest segment of grazers is found in the 18-34 year age group. There are 
also MTV kids who may not see any commercials at all. By ’zoning out', they create a video mosaic, which has no 



theme, but consists of a conglomeration of pretty pictures with no common link. These people are watching 
television, but not attending to a program theme (Evans, 1994). Although, we may not know the precise number 
of commercials that people watch, there is no doubt that the commercial advertisements impact our perceptions and 
attitudes toward others. 

Effects on Children 

The way that people learn about their culture, its values, belief systems, perspectives, and social norms is 
called socialization. Traditionally, socialization was taught by parents, peers, schools, and churches. Over the last 
25 years, the mass media has played a more important role in the socialization process, mainly due to the influence 
of television. A television can be found in nearly every home, requires minimal skill for understanding, is highly 
visual, and is very familiar and appealing to children. Television may play a central role in a child's social life by 
providing common experiences that promote conversations and peer group cohesiveness (Signorielli, 1993). 
Further, children may find endorsements of their self-concept through social experiences. 

There are four types of endorsements that are most likely to affect a child's self-concept. These are personal 
enhancement appeals, social status appeals, product usage portrayals, and competitive product appeals (Adler, et al, 
1980). Each is explained in the following paragraphs. As you read them, consider how television commercials carry 
impact in each area. 

Personal Enhancement 

Personal enhancement appeals range from temporary to more permanent states. The temporary effects 
include such things as fun and adventure while the more permanent states include such things as strength, health, or 
well-being. While all ages of people can be affected by false claims, young children are particularly vulnerable to 
inauthentic portrayals or hypothetical claims made in commercials. A wide assortment of ads for health and beauty 
products, clothing, and foods present a common theme about women’s values. The theme implies that the typical 
woman is preoccupied with the value of beauty, and that she will strive to turn herself into a work of art for the 
promise of self-esteem and popularity with men. In reality many girls can be persuaded by such commercial 
messages. 

A study that exposed one group of high school girls to beauty commercials and another group to neutral 
commercials reported differences in the girls’ beliefs about the value of beauty in terms of being popular with males. 
The girls who viewed the beauty commercials believed that beauty is significantly more important for being popular 
with males than the girls who saw the neutral commercials. In addition, those girls exposed to the beauty 
commercials also rated beauty as more important to them personally than those who saw neutral commercials 
(Signorielli, 1993). 

Social Status Appeals 

Social status appeals imply that ownership of a product will lead to increased social status. Although, the 
National Association of Broadcasting prohibited open display of such appeals, they do exist, at least on a subtle 
basis. These kinds of advertisements typically influence older children for whom peer acceptance is more important 
for their self-concept. Many advertisements promote life styles which the average viewer can only dream about, and 
due to the disparity between the commercials and their own life, may create a sense of powerlessness, frustration, and 
lower self-worth. The affluent life style contrasted with the reality of poverty can lead some people to criminal 
activities, such as stealing or making false benefit claims (Young, 1990). 

Product Usage Portrayals 

Product usage portrayals include exaggerated suggestions of a product's play value or performance 
characteristics. This includes such things as making products appear easier to use than they are in reality. For 
example, the product might be portrayed in a way that appears easy to use, yet children might find it difficult to 
operate. If a child cannot imitate the advertisement and use the product successfully, it may lead to dissatisfaction 
with the product and actually harm a child's self-concept through feelings of frustration and failure. 
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Competitive Product Appeals 

Competitive product appeals include covert competitive claims of implied superiority. These types of 
claims are found in about 11 percent of children’s commercial advertisements, despite the National Association of 
Broadcasting’s prohibition. The quality hierarchy of ’’good-better-best” is changed in advertising to ”good-best- 
better” because it is more difficult for an advertiser to support a claim of ’’better” that one of "best.” Children may 
not be able to discriminate parity claims for what they really are, and they may feel disappointed with presently 
owned products after viewing a commercial that aggrandizes that product (Adler, et al, 1980). 

A study of 250 eighth, ninth, and tenth grade children, found that television viewing was related to beliefs 
about gender roles. After viewing television programming and commercials that showed gender stereotypes, the 
students very more likely to give sexist responses to questions about the nature of males and females, and how they 
are treated by society. Studies have shown that both the commercials and the programming, as separate entities, 
influenced children’s attitudes about gender role stereotypes. Further, there was more evidence of gender stereotyping 
among children who watched a lot of television compared to children who spend a lot of time reading (Signorielli, 
1993). 

It is entirely possible that those children who spend a lot of time watching television have other background 
differences in their lives from those who spend a lot of time reading. Perhaps the social background of their family 
influences the amount of time they spend watching television as well. 

Teaching Television Viewing Skills 

To offset the stereotypes presented by television programming and commercials, it is important to teach 
television viewing skills. Since television is part of everyday life, people need to be become educated consumers so 
that they can live intelligently with the products of science and technology. In other words, they should be informed 
masters of their environment and technology in it rather than naive puppets within the television viewing audience. 

An example of the value of such education is a viewing skills project in Idaho Falls, Idaho. This project 
indicates that critical viewing skills instruction does make a difference in the behavior of children, and basic language 
skills are enhanced in the process (Ploghoft, 1982). There are five basic steps in teaching children to evaluate 
commercials: 

1. Identify what is promised explicitly and implicitly in both the audio and visual portions of the message. 

2. Identify the criteria by which the performance is to be evaluated. 

3. Determine whether the criteria are appropriate to the product or service. 

4. Determine the likelihood of success of the product or service as a solution to the need problem. 

5. Establish the value of the performance of the product or service in terms of the individual. 

Skills such as identifying explicit and implicit promises, differentiating among verbal, nonverbal, and 
visual content, and their intended effects, recognizing persuasive techniques, and comprehending commonly used 
vocabulary are important outcomes in learning to evaluate commercials. In addition, skills in recognizing 
distortions and subtle messages of gender stereotypes are important. 

Individuals who are trained in critical viewing skills will be equipped with criteria for evaluating the 
intention, motives, and audience response to televised messages. They also will be able to assign value or worth to 
the message for any given purpose. These individuals can integrate the message and test it against other information 
bases. They should be able to make inferences and draw logical conclusions (Ploghoft, 1982). With the guidance of 
an adult, children can learn to practice these critical viewing skills on a regular basis and extend that experience to 
reading advertisements within other media as well. Yet despite developing critical viewing skills, children are not 
likely to recognize gender stereotypes if that is what they have come to know. Adults who have knowledge of 
gender stereotyping often do not notice or ignore the instances of it due to accepting it as a part of “the way things 
have always been,” its very subtle innuendo that suggests roles without being noticeable, the attitude of viewing 
television as harmless entertainment, and so on. 

Future of Television Commercials 

Statistics suggest that only about 25 percent of the audience can recall a commercial seen the day before, 
even when given clues, with the exception of the advertisements that are viewed repeatedly and frequently. For 
example, while researching the retention of students who view the Channel One news program, Knupfer found that 
the students remembered nearly 100 percent of the commercials but failed to remember much about the program 
content (Knupfer & Hayes, 1994). This is most likely due the to daily viewing of the program with frequently 
repeated commercials. Indeed, many of the students in that study had the commercials memorized. 
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Several studies have measured recall, comprehension, attitudes, intentions, relevance, believability, and 
likability assuming there would be patterns between these measures and product choice (Stewart, 1986). Yet further 
research suggests that these links are influenced by many factors. Factors that influence recall are not the same as 
factors that influence persuasion. The primary, long-term goal of advertising is to persuade and so it is necessary to 
shift the focus of the advertising effectiveness research so that it will measure persuasion (Stewart, 1986). 

This goal of persuasion can be seen in the rise of infomercials and shopping channels. Larry Grossman, 
former president of NBC News says that television is moving away from the decade of entertainment to the age of 
information (Evans, 1994). Computer technology and interactive television will revolutionize our entire retail 
shopping system. Evidence of this can be seen in the volumes of shoppers who are persuaded to by from such 
television marketers as QVC. 

Information based marketing channels are filling the 500 channel world along with the entertainment 
channels. Channels such as number 397, The Car Buyer’s Channel, are becoming more popular. Upon close 
examination of their target audience, The Car Buyer’s Channel has determined that they should target divorced 
females who are purchasing a car alone for the first time. Such a buyer is hoping to: 

1. Make an educated buy of the best quality and value car 

2. In the most convenient and efficient manner 

3. In the least amount of time 

4. For the smallest amount of money 

Since buying a car usually involves reading advertisements, magazine articles, and other consumer 
information, going from dealership to dealership, test driving, reading product literature, and haggling with sales 
people for days or weeks, the Car Buyer's Channel is offering a new approach. They are using several techniques 
that give the potential buyer a sense of value for their money. Some of these include: 

1 . Product-Specific Long-Form Commercials 

By using 30 minute long commercials about a specific car, advertisers can explain complicated features, 
advantages, and benefits. 

2. Brand/Corporate Image Long-Form Commercials 

Making a particular brand a benchmark for other competitors to be measured against, and instilling brand 
buying such as buying a Volvo for safety reasons are shown. 

3. News 

News events of prices, rollbacks, rebates, engineering, innovations, recalls, and lawsuits are covered 
during newscasts and give the viewer a sense of value for the car they choose. 

4. Model-to-Model Comparisons 

Comparisons are shown as a sort of Consumer Reports magazine. 

5. Interviews and Talk Shows 

Interviews and talk shows can show the viewer what the corporations are really like. "Are they 
environmentally responsible? Testing products on animals? Depleting natural resources or exploiting a 
Third World country? Or are they giving a portion of their profits to a worthy cause? Recycling 
wastes? Donating products to the homeless? Today, who you are is as important as what you make." 
(Evans, 1994). 

6. Short-Form Commercials and Direct Response Offers 

Short two minute commercials are shown during the long-form commercials with call-in offers for 
product brochures, and promotions. Local or regional dealers may market promotions, or car related 
products such as oil or security systems may be shown that may appeal to buyers. 

7. Miscellaneous Programming 

Various programs such as a guide to how well various models hold their resale value, or programs 
which explain financing plans, or call-in shows about specific topics or models are shown to inform the 
viewers. 

Two recent studies by the Opinion Research Corporation have found that programs like the Car Buyer's 
Channel are effective because the information presented is on television and the credibility factor is greater than that 
of newspapers and magazines. They also found that advertising is more believable on television than in the past 
(Evans, 1994). 
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Summary 

Were watching more television than ever, but we're worried about it. We don't like the violence, or we’re 
bored, and so we surf up to five hundred channels, hoping for something better on the next channel. What we find 
along with the programming are commercials, and more commercials, all offering something to provide a quick fix 
to a problem. Leslie Savan concludes that the reason we have a drug problem is because we have an advertising 
culture that encourages the compulsive taking of something (Savan 1994). Further, some researchers say that 
advertising also encourages greed and selfishness, and leads to people being less community-oriented, less 
cooperative, less compassionate, and less charitable (Young, 1990). 

If we become aware of the stereotypes and teach critical viewing skills to our children, perhaps we will 
become informed viewers instead of manipulated consumers. Perhaps we will be able to resist the urge to buy 
compulsively or want something because we are persuaded by advertising that we need it. Perhaps the gender- 
targeted advertisements will have less of an impact on the beliefs, value systems, and self-concepts of people, 
whether male or female. Perhaps we will be able to make different kinds of judgments about whether males need 
hair replacement products or females need low-interest loans to cover the costs of cosmetic surgery. 

Recognition of the blatant and more subtle gender messages is the first step toward resisting the persuasive 
power of gender-stereotyped televised messages. Careful consideration of personal values is perhaps the next. Most 
certainly our young people, whether male or female, need the guidance of parents, teachers, and community leaders to 
help them make well-informed judgments and decisions about the televised gender messages. Perhaps the hardest 
messages to tackle will be those that are subtle, for so often they are dismissed as harmless little irritations. But in 
reality, the constant barrage of subtle messages can be just as persuasive and influential as the more blatant 
messages, thus perpetuating images that people cannot live up to and most certainly should not be restricted to. 
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Confronting recent design challenges, instructional designers have latched onto adaptive procedural 
techniques from outside the Instructional Systems Design (ISD) field. One of the most frequently mentioned is 
"rapid prototyping" (e.g., Jones, Li, & Merrill, 1992; Tessmer, 1994; Tripp & Bichelmeyer, 1990), the iterative 
creation of partial products over the course of design and development. The importance of the notion of rapid 
prototyping to our field is not, however, as a technique to be adopted, but as a vehicle for understanding the 
instructional design/development process. Rapid prototyping (RP) brings to the foreground key issues often passed 
over in the discussion of instructional design models: the designer as inquirer and facilitator, the importance of 
knowledge derived “ in ” doing, and the seldom-examined assumptions underlying a “ scientific ” approach to design. 

Prototype as the Designer’s Cognitive Tool 

In rapid prototyping, designers bring a product into being through the creation of successive prototypes. 
The intermediate prototypes become an important means of getting feedback; the design and development process 
become intertwined. Rapid prototyping "provides the designer with concrete feedback in terms of the final product, as 
compared to the more abstract feedback provided by the standard products of analysis and design." (Jones, Li, & 
Merrill; 1992). Below the surface, however, rapid prototyping seems to support two very diverse orientations to 
design. The purpose of this section is to discriminate between two views of rapid prototyping — one in which 
prototyping is a development technique and one in which rapid prototyping represents a distinct way of knowing — 
and then to suggest how viewing the prototype as a tool-for-thought challenges the design/development hierarchy 
reflected in instructional design models. 

While rapid prototyping is said to offer more opportunities for feedback, earlier in the development process, 
exactly what constitutes feedback is not always specified. It’s easy to think of feedback in instrumental terms: does 
the prototype work? Did it achieve the desired effect? However, we discriminate between two kinds of feedback, 
feedback gained from testers and feedback used by the designers themselves. In the case of the former, getting 
feedback is very much like a formative evaluation, defined by Dick and Carey (1996) as "the collection of data and 
information during the development of instruction which can be used to improve the effectiveness of the 
instruction.” In the second case, however, when the prototype “talks back” to the designers themselves, the prototype 
being tested becomes a tool for thought and critical reflection. 

The idea of the prototype as an intimate mediator of immediate feedback follows philosopher Larry 
Briskman's work (1981) on creativity in science and art. He suggests that intermediary products (i.e., prototypes) are 
important because they serve as aids to the reflection/creation process. He writes: 

The artist must build up his painting gradually, stroke by stroke; while the 
theoretician must build up his conjectural explanation bit by bit (even though he 
may have got his explanatory 'core idea* in a flash). But if this is the case, then 
it is highly likely that the very thought processes of the artist or scientist will 
themselves be affected by the work done so far. In other words, the creator, in his 
very process of creation, is constantly interacting with his own products; and 
this interaction is one of genuine feedback, for the creator is as much influenced 
by his own initial creations as these were influenced by him.... If all this is 
correct, then it follows that we shall not be able to describe, let alone understand, 
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the creative process unless we make reference to the "intermediary" products 
which function, so to speak, as "tools” in the creative process itself. 

The notion of intermediary products as tool s-for* thought is important because it points to a particular 
vision of rapid prototyping. Rapid prototyping is sometimes described as a process of continual refinement in which 
a product is brought closer and closer to its final realization. Certainly this is a valid and useful conception. 
However, thinking of rapid prototyping as a partner in thought enables us to conceive of the prototype in another 
role. As well as being a step on the way to a final product, each prototype becomes a means of investigation, a 
cognitive tool. 

Seeing a prototype as a cognitive tool questions some of the central metaphors of the development process. 
Michael Reddy (1993) has described how the English language is filled with metaphors which imply that "language 
functions like a conduit, transferring thoughts bodily from one person to another." For instance, we are taught to 
"put ideas into words." Sentences may be "empty of both meaning and feeling." Language is seen as a container or 
conduit for thought. The development process in traditional instructional design is often thought of in much the 
same way. The instructional design is a blueprint which development brings into a tangible form. A hierarchical 
relationship is created in which concept is prized at the expense of the concrete; instantiation becomes a mere 
container for strategy. Seeing a rapid prototype as a tool for discovery challenges this hierarchy. The act of 
conceptualizing is integrated into the act of practice-the act of doing is the act of thinking. This view of rapid 
prototyping is based on a praxis orientation to design. Praxis thought affirms the importance of an “engagment with 
materials", and most importantly, promotes deliberation about purposes as well as about means (Craig & Tracy, 
1995). From the praxis orientation, rapid prototyping is much more than a method for collecting information. It 
embodies a particular kind of thought process that occurs in interaction with materials. 

This alternative to formal, propositional reasoning is frequently described as a "concrete approach" to 
knowing. It is tactile and contextual. Turkle and Papert (1990) have used Levi-Strauss’s term "bricolage" to describe 
concrete knowing. Levi-Strauss associated what he also called the “science of the concrete” with primitive societies. 
Piaget saw bricolage as typical of a developmental stage in children. Writers such as Gilligan, Belenky, and Keller 
(cited in Turkle and Papert, 1990), have identified the concrete approach to knowing as characteristic of many 
women. Until recently, the "bricoleur" knower seemed inferior to the "formal" knower. The revelation that important 
scientific discoveries begin with down-to-earth, concrete, tactile experimentation has done much to elevate concrete 
knowing.. 1 No longer a developmental stage, a primitive form of science, or an “essentially” feminine characteristic, 
bricolage is now considered a legitimate style of reasoning. Paradoxically the computer, a product of formal logic and 
propositional knowing, by facilitating rapid prototyping, has helped revaluate concrete approaches to knowing by 
giving people access to and experience of them (Turkle and Papert, 1990). When development is seen as “concrete 
knowing” or “bricolage”, development can be elevated to the status that design now holds in the traditional view of 
ISD as “design, then development”. The designer, in the act of doing, uses the prototype as a tool of inquiry; 
development is not mere execution. The distinction between design and development is blurred; the hierarchical 
relationship is equalized. 

The Designer as Co-inquirer 

ISD is not a process in which the designer is left alone with his or her materials. It is a social process as 
well. If rapid prototyping is to act as a vehicle for understanding the instructional design/development process then 
we must examine RP in the context of how it is positioned and used by a group. Other people join the designer's 
"conversation with materials”, adding their own conversations and their communications with each other. 

When rapid prototyping is viewed primarily as an inquiry tool that may be used collaboratively with non- 
designers, the designer's priority and responsibility is to support collaborative investigation. The prototype helps the 
designer and her collaborators create the proper environment in two ways: by helping the collaborative group "reflect- 
in-action" (Schon, 1983) and by serving as a focal point for development of a project culture. The shift in the 
designer's role from technical specialist to co-inquirer is not, however, simple to make, and this section suggests 
some of the paradoxes and additional obligations that the paradigm shift presents to the professional designer. 



1 For example, Nobel Laureate Barbara McLintock’s acknowledgement that a “feeling for the organism” has 
been essential to her scientific discoveries (Keller, 1983, cited in Turkle & Papert, 1990). 
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Based on their work designing an electronic learning and information environment for AT&T, and on their 
work using prototyping and participatory design techniques in instructional design at Indiana University, Dorsey, 
Goodrum and Schwen (in press) put forward a view of rapid collaborative prototyping as a paradigm for instructional 
development. Contained within this view is a brief description of how the paradigm changes the relationship between 
the user and the designer, the designer's primary responsibilities, and the nature of the designer’s expertise: 

In a collaborative prototyping model, the user 'uses' the designer as an 
information resource and facilitator of a design process to solve their 
instructional need. Emphasis in designer responsibility and expertise shifts more 
to project management, establishing realistic expectations, facilitation of 
structured meetings, creative brainstorming, and evaluation. 

Coyne and Snodgrass (1993) suggest how collaboration transforms traditional definitions of expertise. They 
conclude that one must work against the overwhelming legacy of the Cartesian rationalist and Romantic 
philosophical traditions, which both glorify the individual, in order understand oneself in relation to society; both 
traditions "militate against cooperation". Coyne and Snodgrass arrive at a redefinition of professionalism and 
expertise in the general context of design. Expertise is seen aa a matter of skill in facilitating and managing the 
collaborative process, "not as an objective outsider but as a participant." Expertise resides within a community and 
makes sense only in relation to the other expertise brought into the design process. "Positing design ideas is an act 
of self disclosure, projecting expectations into a conversational domain, illuminating prejudices, challenging norms, 
and bringing about meaningful action" (Coyne and Snodgrass, 1993). 

The forgoing re-formulation of designer's position within the design process asks instructional designers to 
reflect on the actions they take in collaboration with others. However, just as the distinction between the use of a 
prototype to refine (rationalistic orientation) and the use of a prototype as a way to gain knowledge in doing (praxis 
orientation) has been made, we can more clearly make the distinction between reflection on action involving others 
and reflection in action with others. 

The highly iterative nature of rapid prototyping (RP) involves frequent evaluation and re-visiting of the 
problems addressed and created by the product under development. RP can facilitate what Schon (1983) calls 
"reflection-in-action" which he says is vital to coping with particularly troublesome situations. In a collaborative 
view of rapid prototyping, the group builds and responds to the mockup or prototype. The mockup or prototype is 
the focal point for the group's critique, idea generation, and idea testing. The group’s conversations or interactions 
with the prototype can be extremely varied in quality and intensity. The designer faces a moment to moment 
challenge of balancing the free rein of ideas and critical judgments with bringing about progress. 

Schrage has aptly called the design product "congealed culture" (1993). Through the collaborative building 
of concrete objects-plans, prototypes-shared language and artifacts develop, central to the development of a unique 
project culture. "The instructional designer must be sensitive to the meanings that are constructed collaboratively 
within the larger culture of the project and the smaller culture of the design team" (Campbell-Bonar and Olson, 
1992). The product serves as a means of both defining and promoting the negotiation that takes place during the 
process. When products are brought into being early in the design process they serve as a shared focal point for 
reliable communication and knowledge generation as well as provide a structure and a vocabulary for that 
communication. It then becomes the designer's responsibility to consciously and explicitly use the prototype as a 
means to insure that two levels of need are met: problem-solving and setting at one level and the political and 
interpersonal needs of the developing collaborative design group at another. Schon and Rein (1994) call this "double- 
designing". 

Collaborative rapid prototyping suggests that collaboration is more than an alternative design technique, 
another way to organize the work of others. It is also a way of being and knowing, an alternate vision of 
professional identity and practice. When the prototype is viewed as a tool of inquiry used in collaboration with 
others, the designer publicly throws off the mantle of the expert admitting her lack of cultural and problem 
knowledge and valuing the cultural and problem knowledge of others. But while the image of the collaborative 
inquirer is often encouraged and glorified, living up to the image can be difficult, as pointed out above, especially 
when the cultural context of instructional design and development is so heavily rooted in a positivistic, rationalist 
approach to science. Further, questions arise as to whether the instructional products of collaborative inquiry are 
superior to those created in a more traditional way, and as to whether just as successful instructional products might 
be created by designers with a more traditional image of their profession. Again, however, we return to the point that 
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collaborative rapid prototyping is not an alternative set of techniques; our purpose here is to make a legitimate place 
in the field of instructional design for an alternative view of the designer, practice, and the essential nature of the 
relationship between design and science. 

Practitioner as Producer of Knowledge 

The theme running throughout this paper with a starting point of rapid prototyping, is the 
conceptualization of Instructional Systems Design as inquiry. One way to examine design as inquiry is to ask if the 
practice of Instructional Design is a science. 

Whether referred to as Instructional Design (ID), or its traditional, procedural Instructional Systems Design 
(ISD) model, it is not particularly difficult, or rare, to imagine practice within the field or its components as not 
being a science. Davies (1991) for a commonly cited example, eloquently argues that ID is definitely an 'art' in 
which craft and science might play a role. If one were to ask practitioners if they were 'scientists', they would 
probably answer 'no'. They label themselves 'designers' or 'consultants’ or perhaps 'technologists' (an exception 
might be those in academia some of whom might call themselves researchers'). In recent discussions about 
alternatives to traditional ID practice, there is a linking to other fields of design, such as architecture, software 
design, and so on (e.g., Tripp & Bichelmeyer, 1990; Rowland, 1994; Tessmer & Wedman, 1995; Winer & Vazquez- 
Abed, 1995) and a contrasting of design with science. 

There are a number of ‘scientific’ labels that are often applied to ID — the scientific approach, the systems 
approach, the learning theory base, and so on — but the actual practice of ID is frequently considered something else, 
either an art or a craft. This may be because, as a field, ID considers itself a fairly young discipline and recognizes it 
still has a lot to figure out. Much, if not most, of ID has not been expressed as 'natural law' or 'empirically tested 
theory' or even 'established fact', though there is a generally accepted mode of practice referred to as the 'systematic 
approach'. Furthermore, it's hard not to recognize the contributions that aesthetic judgments and even personal fiat, 
for example, contribute to a final product. But perhaps more importantly, the practice of ID is not generally 
considered a science because it does not seem to conform to a traditionally held view of what scientific practice is: It 
is difficult to consider the average ID project as a rigorously controlled experiment or even an in-depth naturalistic 
case-study. 

The prevailing notion is that through one avenue or another, some scientific basis of ID is generally 
assumed, but the practice of ID is not a kind of science itself. 2 Within such a view, the practice of ID is considered 
separate from both the theory generation and the controlled experimentation of science. Practitioners are the 
consumers, not the producers of 'science'; even the academics in our field have been characterized as consumers of 
research (Hannafin & Hannafin, 1991). As consumers, practitioners are led to depend on and follow the results of 
science. In other words, the practitioner should confidently proceed in the practice of ID because it is based on the 
results of other , scientific, work. 

Applying the critical rational approach, espoused by the philosopher Sir Karl Popper, supplies a promising 
alternative to prevailing views of design practice that could include design within a liberal conception of scientific 
inquiry. A Popperian approach both resonates well with the vision of designing and designers presented in this paper 
and offers a path to the additional rigor desired by those conducting inquiry. The design inquirer searches for 
disconfirming evidence about the design or the design. It is through intending to root out error rather than to seek 
confirmation of a solution that design can move towards meeting the standards of rigor of the critical rational 
approach. 

For Popper (1985; 1972), inquiry (i.e., effective thinking) is evolutionary and iterative. It starts with Pj f 
the identification of a problem; TSj , a tentative solution (hypothesis or plan for action), EE , error elimination by 
severe critical examination through practical or theoretical testing, and > the new problem situation as it emerges. 

Pj => TSj => EE => P 2 



2 There is often a general distinction made between practice and science, where technology is the application 
of scientific discoveries, though it is argued that this idea is no longer in vogue among historians (Schwartz, 1992, 
cited in Seels & Richey, 1994). 
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But in this form, a serious aspect is still missing: a multiplicity of tentative solutions, a multiplicity of severe 
testings, a multiplicity of formulations about the problem situation. A final schema would look something like 
this: 



/ TS,N * 

\ => TS 2 => EE 

v ■/ 



(Popper, 1972, p. 243) 

Popper’s focus is on the building of knowledge through the elimination of error — both in our 
conceptualization of problems as well as our tentative solutions to them — through critical examination. In 
practice, this means, for example, placing a prototype, a tentative solution to a problem in a particular setting, under 
the most severe scrutiny of others, under the worst possible conditions, in the most hostile environment. One wants 
to know, in effect, what will break it, undermine it, destroy it, make it unworkable. If one tests a prototype only 
under the best of conditions, under the scrutiny of like-minded people, and so on, one is only looking for confirming 
evidence. 

For Popper, the way to practical results, to find out whether some action “is likely to produce an expected, 
or desired result”, is through “piecemeal tinkering” and critical analysis (Popper, 1985, p. 304). Popper favors a 
piecemeal approach for fundamentally ethical reasons: With the piecemeal approach a person conducting inquiry can 
approach the scope of the human change with an open mind, keeping a close eye on the range of unintended 
consequences and thus imposing a “discipline on our speculative inclinations” (Popper, 1985, p. 305). In short, a 
critical rational approach could be quite helpful in connecting (or, if you will, reconnecting) the practice of 
instructional design directly to rigorous inquiry, with the designer becoming a producer and contributor of knowledge 
in the field. 

Conclusion 

A design paradigm specifies [...] the essential character of the design domain and 
proposes a set of design principles for working in that domain, utilizing a set of 
technologies, tools, and methods. A design paradigm prescribes values by 
highlighting the most important aspects of the design domain. Further, a design 
paradigm is motivated by a philosophical orientation and relies upon a set of 
relevant scientific principles. (Moran and Anderson, 1990, p. 383) 

Our own day-to-day use of a particular technique, rapid prototyping, combined with thoughtful consideration of its 
impact — on our practice and on the experiences of the people we work with and for — has engendered our vision of 
a new paradigm for ISD. This picture of the paradigm is not complete, but we have sketched, we believe, key 
elements of it. The philosophical orientation of the paradigm is toward “practitioner as inquirer.” Designers are 
valued not only for their own innovative response to unique situations, but for their ability to facilitate collaborative 
inquiry. The paradigm’s relevant scientific principles arise from conscious problematizing in the course of practice. 
That these ideas have sprung from our practice lends further support for the notion of design as inquiry. 




29 f 



r W KJ 



0 



References 

Briskman, L. (1981). Creative product and creative process in science and art. In D. Dutton and M. Krausz 
(Eds.), The concept of creativity in science and art. Boston: Martinus Nijhoff. 

Campbell-Bonar, K. & Olson, A. T. (1992). Collaborative instructional design as culture-building. 
Canadian Journal of Educational Communications ; 27(2), 141-152. 

Coyne, R. (1995). Designing Information Technology in the Postmodern Age: From Method to Metaphor. 
Cambridge, MA: The MIT Press. 

Coyne, R. & Snodgrass, A. (1993). Cooperation and individualism in design. Environment and Planning 
B: Planning and Design , 20, 163-174. 

Craig, R. T. & Tracy, K. (1995). Grounded practical theory: The case of intellectual discussion. 
Communication Theory, 5 (3), 248-272. 

Davies, I.K. (1991). Instructional development as an art: One of the three faces of ID. in D. Hlynka and J. 
C. Bellands (Eds.), Paradigms regained: The uses of illuminative, semiotic, and post-modern criticism as modes of 
inquiry in educational technology. Englewood Cliffs, NJ: Educational Technology Publications, 1991, 93-106. 

Dick, W. and Carey, L. (1996). The systematic design of instruction , 4th ed. New York: Harper Collins 
College Publishers. 

Dorsey, L., Goodrum, D. A. & Schwen, T. (in press). Rapid collaborative prototyping as an instructional 
development paradigm. In C. Dills & A. Romiszowski (Eds.), Instructional Development: The State of the Art, III. 

The paradigms, models, metaphors & viewpoints. Englewood Cliffs, New Jersey: Educational Technology 
Publications. 

Hannafin, M. J., & Hannafin, K. M. (1991). The status and future of research in instructional design and 
technology revisited. In G. J. Anglin (Ed.), Insdtructional technology : past, present, and future, 302-309. 
Englewood, CO: Libraries Unlimited. 

Jones, M. K., Li, Z., & Merrill, D. M. (1992). Rapid prototyping in automated instructional design. 
Educational Technology, Research, and Development, 40(A), 95-100. 

Moran, T. P., & Anderson, R. J. (1990). The workaday world as a paradigm for CSCW design. Proceedings 
of the Concference on Computer- Supported Cooperative Work, October, 1990, Los Angeles, CA, p. 381-393. New 
York: ACM. 

Popper, K. R. (1972). Objective knowledge: An evolutionary approach. London: Oxford University Press. 
Popper, K. R. (1985). Popper selections. D. Miller (Ed.). Princeton, N.J.: Princeton University Press. 
Reddy, M. J. (1993). The conduit metaphor: a case of frame conflict in our language about language. In A. 
Ortony (Ed.), Metaphor and Thought. Cambridge: Cambridge University Press. 

Rowland, G. (1994). It’s the sound! Preface to the special issue on designing for human performance. 
Performance Improvement Quarterly, 7(3), 3-6. 

Schon, D.A. and Rein, M. (1994). Frame reflection : toward the resolution of intractable policy 
controversies. New York: Basic Books. 

Seels, B. B. & Richey, R. C. (1994). Instructional technology: the definition and domains of the field. 
Washington, DC: Association for Educational Communications and Technology. 

Schrage, M. (1993). The culture(s) of prototyping. Design Management Journal, Winter, 55-65. 

Tessmer, M. (1994). Formative evaluation alternatives. Performance Improvement Quarterly, 7(1), 3-18. 
Tessmer, M. & Wedman, J. (1995). Context-sensitive instructional design models: A response to design 
research, studies, and criticism. Performance Improvement Quarterly, £(3), 38-54. 

Tripp, S. & Bichelmeyer, B. (1990). Rapid prototyping: An alternative instructional design strategy. 
Educational Technology Research & Development, 3&(1), 31-44. 

Turkle, S. & Papert, S. (1990). Epistemological Pluralism: Styles and voices within the computer culture. 
Signs, 76(1), 128-157. 

Winer, L. & V&zquez-Abed, J. (1995). The present and future of ID practice. Performance Improvement 
Quarterly, 8(3), 55-67. 




296 



291 



An ISD Model for Building Online Communities: Furthering the 

Dialogue 

Jason Ravitz 
Syracuse University 



Abstract 

This model explains the people , processes, and products that are involved in creating online learning 
communities for K- 1 2 students and teachers. It is hoped that by organizing and making explicit what is done, the 
level of discussion will be raised and practice will be improved. The purpose is to help those who would engineer 
the process, analyze it, shape it, and evaluate it in order to provide better educational experiences for learners. This 
paper reviews research and theory from both constructivist and ISD literature, arguing that a model that draws on the 
strengths of both perspectives is preferred The product of the model is a sustained learning community, and an 
active network of people conducting projects using networked connections and resources. 

Introduction 

This paper was originally written in response to a special issue of Educational Technology (Sept-Oct, 1995) 
entitled "Constructivist Learning Environments" that highlighted the tension between educational researchers 
conducting constructivist learning experiments and the Instructional Systems Design (ISD) community. Lin et al. 
(1995) provide an invitation to dialogue, arguing that such an effort is worth undertaking. 

The author has had the opportunity to work with constructivist educational researchers outside the field of 
ISD. Some of these researchers see ISD-type programs as being on the "wrong side of a paradigm shift". This paper 
examines the knowledge that is growing around the use of networks for collaborative learning, and the role that ISD 
processes can play in advancing this work. It hopes to contribute to the dialogue proposed by Lin et al. (1995) by 
discussing some of the strengths of the ISD approach. In order to do this, this paper attempts to do the following: 

1. Share insights into the development of online collaborative learning experiments 

2. Discuss the role of ISD, challenges, and how ISD can be relevant 

3. Present examples of design components within an ISD-based framework 

Wilson, et al. write, "The literature on constructivism is filled with theoretical dialogue but few design 
models or concrete suggestions for practice". In their article, Lin, et al., argue that "a structure for organization and 
management could be of great benefit to those attempting to implement the idea of learning communities" (Lin et 
al., 1995). ISD seems well suited to provide this structure. 

People trained in ISD have unique talents to bring to bear on the development of collaborative experiments. 
According to Lin, et. al., "A strength of the instructional design community is its ability to articulate, manage, and 
systematize the process involved in designing effective learning environments" (p. 59). 

Educational Context: A new paradigm? 

There is a growing body of literature that discusses using computer networks as an interactive 
communication-rich environment to foster collaboration and shared construction of knowledge (Fishman and Pea, 
1994; Gordin, et al., 1994; Gomez, et al., 1994; Hunter, 1993; Koschmann, et al., 1993; Ravitz, 1995; Riel, 1989; 
Romiszowski and Ravitz, 1997). To the extent that these networks are used to form partnerships for social and 
educational experiences beyond the classroom, one sees the development of a "unique model of network learning" 
(Riel, 1994). 

For developers, the Internet has helped to bring about a shift from an "instructional" model to an 
information-age "conversational" model of learning (Chang and Romiszowski, 1994; Reigeluth, 1994; Romiszowski 
and Ravitz, 1997). Jonassen (1995) notes that while new technologies can make individualized learning more 
powerful, they can also be used by constructivists "to support conversation among communities of practitioners and 
learners". For leaders of this movement, it is necessary to reconceptualize "the computer as a knowledge 
presentation device to one that supports a pedagogical focus on communications in support of collaborative learning" 
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(Koschmann et al., 1993). Fishman and Pea (1994) write that "the network's true power comes from the synergy of 
many dispersed minds working together to solve problems and discuss issues." 

This context for this model involves the creation of environments that support collaboration and 
cooperative (Heinich, Molenda et al., 1996, p. 322) learning experiences. Learning is seen as taking place through 
the social construction of knowledge with authentic activities providing greater motivation and meaning to students 
(Collins, 1994; Riel, 1994; Riel and Levin, 1990). Furthermore, a more active role for teachers and students is 
being sought through the "participatory-design" of instruction (Silva and Breuleux, 1994). Instead of being overly 
prescriptive, teachers and students "help define or select content and design their own learning experiences" (Wilson, 
et al, 1995). In sum, the purpose of this model is not to deliver instruction, but to foster meaningful learning 
experiences. 

Challenges for ISD 

Some problems with the traditional ISD approach have been noted in the literature. It is clear that online 
learning communities cannot be pre-packaged and delivered. Instead, constructivist projects may require that "all 
major constituencies be represented on the design team, including teachers and students" (Wilson, et al., 1995). 
Moreover, this adaptation must occur at each site where implementation occurs. "Effective learning communities 
must be reinvented from location to location rather than simply transported and implemented" (Lin, et al., p. 58). 
Perhaps the greatest challenge, then, is to view ISD as less deterministic (Rowland, 1995, p. 21), particularly in 
terms of learning outcomes. Andrews and Goodson (1980) acknowledged that discovery learning "might reject the 
specification of objectives and corresponding direct match of instructional events to these objectives" (p. 13). 
Constructivist theorists extend this, suggesting that it is often appropriate for students to undertake engaging tasks, 
and "let any facts and concepts be learned incidentally" (Collins, 1994). 

Indeed, some might argue that the development of non-determini Stic learning experiences is not 
instructional design! Before this work is dismissed as "not instructional design" it is important to consider the costs 
to the field, and individuals in the field, that this attitude might cause. The risk is that non-ISD researchers will 
press ahead, while students of ISD will not be supported and will be alienated from the work of would-be colleagues. 
The model that follows draws upon the strengths of ISD, the management and process-based expertise that ISD 
offers, and places development activities within an ISD framework. 

The Model — developers, learning tasks, and procedures 

What is presented here is a non-deterministic approach to instruction that involves teachers, learners and 
mentors in the process of developing and delivering instruction. The developers in this model are people seeking to 
build connections between K-12 teachers and students with people and resources available outside the classroom - 
other schools, museums, community centers, universities and research centers. 

The assumption is that the developers are: 

1) using Internet-based technologies 

2) to link classrooms with outside people, communities, resources, and organizations 

3) in order to support authentic, collaborative, network-based projects 

4) by "early adopters" — people ready, willing and able to participate in online collaboration, and 

5) using an ISD approach (when appropriate) to inform the design, development and evaluation process. 

A successful implementation of the model would deliver a lasting resource that supports many types of 
learners and learning activities. It would provide the following: 

• resources and support for teachers 

• implementation in the classroom, with the system extending outside 

• interaction between students and teachers with peers and mentors 

• geographically distributed participants 

• electronic network-based and facilitated conversations 

• communication intensive environments 

• flexible means and ends 
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Illustrative Examples 

Two example development efforts are used to illustrate the proposed model. 

1) Learning Community: Focus Promised Land is a pilot effort sponsored by Discovery Communications, 
Inc. and undertaken with the support of Interactive Frameworks, Inc., (then Duggan Associates) and AskERIC at 
Syracuse University. The author served as a design consultant on the AskERIC team. William Duggan from 
Interactive Frameworks, Inc. was helpful in contributing ideas to this paper. The Promised Land (PL) was 
undertaken to advance opportunities for rich learning experiences via the Internet and to support ongoing exploration 
by students and teachers concerning issues raised in the televised series of the same name. The televised series 
involved the experience of African Americans who migrated earlier in this century from the South to northern US 
cities. Numerous schools already receive Learning Channel broadcasts with a pre-packaged curriculum guide. For 
this pilot, those with Internet access were invited to participate online after viewing the televised series. Teachers 
were asked to join a Listserv and visit the PL Web site in order to share ideas and to begin developing projects with 
their students. Ultimately, classrooms were encouraged to propose, develop, and carry out projects while taking 
advantage of the availability of online resources, facilitators and mentors. The goal was to support teacher 
innovation, develop meaningful student learning, and foster reflection in the local community and via Internet-based 
communications. For more information see: — http://school.discovery.com/leamingcomm/promisedland/. 

2) The Online Internet Institute is a collaborative online experiment for the professional development of 
teachers first sponsored by the National Science Foundation (NSF Grant No.REC-9554232). The Online Internet 
Institute (OH) was designed to support teachers' use of the Internet. It combines regional face-to-face learning with 
the opportunity to experience online collaboration first hand. Small groups of individuals collaborate and share with 
the online education community. Information on their experiences and the understandings they gain are collected and 
made available online. The author has served OH as a designer and evaluator. For information see: — http://oii.org. 

Margaret Riel's work on the AT&T Learning Circles Teacher’s Guide (http://www.att.com/ 
education/lcguide/) is also provided as an example for consideration. While the author was not involved in the 
development of this work, and therefore has few insights into the development process, some useful structures are 
provided that might be incorporated into this model. It is now available through the International Education and 
Resource Network ( I*EARN). For more information see: — http://www.ieam.org/ieam/circles/lc-home.html. 

Types of Learning Tasks 

It should be noted that the type of learning tasks one sees on the Internet varies greatly (Eisenberg and Ely, 
1990, Harris, 1994; Levin, et al, 1989). What is important "regardless of the technology selected, is to connect 
technology to powerful learning paradigms" (Jones, 1994, p. 57). Some of the most powerful Internet-based 
projects take advantage of a unifying theme (such as an historical event) to draw in the interests of diverse people. 
Other projects are designed to share data and resources (Gordin, et al. 1994), or to build on a shared experience or 
purpose (Riel and Levin, 1990). 

Such designs ideally allow students and teachers to (1) work on authentic, meaningful, and 
challenging problems; (2) interact with the data in user friendly ways that allow some student 
control of learning. (3) build knowledge together within a learning community that is broader than 
a few students or schools with similar characteristics and interests; and (4) interact with practicing 
professionals and community members (Jones, p. 57). 

The focus of this model is to support local participants as they develop activities that are meaningful to 
them. For example, taking advantage of geographic diversity, participants can share online that which is unique in 
their communities. PL participants shared locally produced recipes, worked with local artists and musicians, 
collected oral histories of community members, and discussed population demographics. OH collaboration was based 
on the shared issues and concerns of teachers as they attempted to learn about and integrate the Internet into their own 
teaching. In both cases, individuals develop and share their work across distance and time, potentially benefiting 
from the experiences of others. Thus, based on the above discussion, this model addresses the aspects of instruction 
that were described by Lin., et al (1995): 

The nature of efficient learning communities can be summarized as providing students 
opportunities to: 



299 

294 



1) plan, organize, monitor, and revise their own research and problem solving; 

2) work collaboratively and take advantage of distributed expertise from the community to allow 
diversity, creativity, and flexibility in learning; 

3) learn self-selected topics and identify their own issues that are related to the problem-based 
anchors and then identify relevant resources; 

4) use various technologies to build their own knowledge rather than using the technologies as 
'knowledge tellers'; and 

5) make students' thinking visible so that they can revise their own thoughts, assumptions, and 
arguments" (p. 59). 

Procedural description of the Model 

Before work can begin, a substantial commitment must be made to a large-scale development effort. 
Because of the resource-intensive nature of this work, it is doubtful that individuals teachers or students could hope 
to undertake a development effort of this nature. Both PL and OH relied on the expertise of services such as 
AskERIC which developed the PL web pages and provided the technological tools required for online collaboration. 
Oil benefited from a server and a threaded, web-based discussion environment supported by -BBN Systems and 
Technologies via another NSF-funded project. 

There are guidelines elsewhere in the literature of how teachers can develop collaborative projects for then- 
own students using the Internet (Harris, 1994; Harris, 1995; Levin, et al., 1989; Riel, 1993; Rogers, et al, 1990). 
These guidelines (see http://www.ed.uiuc.edu/Guidelines/) are considered important for this model. The rest of this 
paper discusses the development of learning communities intended to support many simultaneous projects. It 
employs the common stages of ISD: management, front end analysis, design, development, implementation, 
evaluation and dissemination. By attempting to integrate this work into an ISD framework the author recognizes the 
benefits provided by ISD processes. In this case, the result is a process, or system for learning rather than an 
instructional product (Gustafson and Powell, 1991, p. 7). 

1. Management (creating the development team) 

Management of people and processes is a central component of the model. Oil started with a face-to-face 
meeting of two dozen individuals including staff, design consultants, partner organizations, mentors, and 
participating teachers. Subsequent gatherings have taken place at national and state educational conferences where 
key issues are addressed. PL started with discussions between Discovery Communications, Duggan Associates and 
AskERIC. 

Interactive environments enabled these teams — developers, mentors, teachers, service and resource providers 
-- to improve the process, features, and services being developed. Working in an online environment offers 
opportunities for better coordination, and the interactivity of the medium can provide a unique level of management 
responsiveness, especially important at the pilot stage of development. Nonetheless, the complexity of the task is 
daunting. Strategies are lacking for "managing a large number of human resources and bringing them together to 
reach consensus on their goals" (Lin et al., p. 60). Once participants become involved, it becomes necessary to 
have a facilitator in place to "take on the responsibility of monitoring and facilitating the group interactions" (Riel 
and Levin, p. 164). This role should be defined early in the process and performed by a member of the development 
team, or someone who works very closely with the developers. OH had one or two staffers who served as facilitators 
and subsequently tried to delegate responsibilities to a site facilitator in each school or district that joined. For PL, 
the developers served as online facilitators, while the classroom teacher or local mentors could be thought of as the 
local facilitators, monitoring and supporting the work of students. 

2. Front End Analysis (selecting a theme, identifying resources) 

It is necessary to establish the basis for this work in terms of meeting the needs of participants. PL started 
with a central theme. The people who became involved were those who saw opportunities to make the project 
meaningful to them. In contrast, Oil teachers began with the problems they wanted to address as learners, then 
sought out resources, mentors, and collaborators. In both cases, the core idea around which each project is 
constructed is something of value to the participants and their local communities. 

Early in the development process it is necessary to identify resources that are available. One of the 
strengths of using the Internet is that existing projects, mentors, subject matter experts, and organizations can be 
made available to support investigations. It is the responsibility of the developer to initially evaluate these 
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resources, and foster relationships with individuals or organizations who may have increased demands placed on them 
as a result of participation. Resources include online, regional, and local data sources, as well as human resources. 
The Promised Land secured subject matter experts (SMEs) in communities where schools were active to help 
students and teachers develop their projects in different subject areas. "SMEs can help design learning experiences: 
designers manage projects, build teams, check for content and accuracy, and serve as model learners and teachers." 
(Wilson, et al, 1995). PL also provided access to people and content used in the production of the series; the narrator 
was available to teachers and students, in addition to scripts, transcribed interviews, music and video footage, and 
photographs that were used in the series. Oil started with a cadre of mentors to support teachers' investigations. 

3. Design (providing communications tools, identifying shared interests and teams) 

In a constructivist model, the central role of the developer shifts from creator of instruction to manager of 
processes (Wilson, et al., 1995). Part of the design process includes the development of communication channels to 
support exchange of ideas, e.g., via Listservs. The design must address the process and means by which participants 
will communicate with one another and find shared interests. Riel and Levin (1994) refer to this as providing 
"response opportunities" to all participants. 

There must be a basis for interaction because the model assumes no prior knowledge between participants. 
"Student participants on networks rarely begin their interaction with any knowledge of each other" (Riel and Levin, 
p. 162). For PL participants, a shared experience was provided by the televised series. For On participants, groups 
were formed based on shared learning needs, e.g., identifying quality resources. 

The design must afford opportunities for matching mentors, resources, and participants and fostering the 
development of relationships; this includes mechanisms for collecting, synthesizing, and sharing data provided by 
participants about their interests. PL relied heavily on the facilitators to help build these connections, while OH 
experimented with automated ways of providing information and making connections via a shared database. By 
providing detailed information about all participants, strong "virtual" groups were formed within Oil that had almost 
no face-to-face contact but developed valuable products nonetheless, e.g., a downloadable tutorial. 

4. Development (develop projects) 

During this stage, participants are assisted in the development of projects based on their interests and 
groups. PL developed a unique approach called "toe-wade-dive" to support project development: 

a) toe — participants share their project ideas and receive feedback and suggestions from interested parties; 

b) wade — participants share their plans for developing the project; 

c) dive — participants announce their intention to go forward when they are ready 

The PL approach permits projects to start and end according to their own timeline. This is an important 
trade-off to consider, while some projects may benefit from a fixed timeline for all participants, particularly one that 
is sensitive to the school calendar (Rogers, et al., 1990). 

Throughout the development process, participants are encouraged by the facilitator to interact with resource 
people, mentors and each other. While their activities may be very different, each group should share its planning 
with others. Communication during project development offers opportunities for learning — mentors may be 
involved throughout the process to "confront learners' naive, intuitive theories and to scaffold their learning" 
(Wilson, et al., 1995). The presence of others may offer teachers assistance, including discussion of pedagogical 
issues, e.g., how designing the project can promote the development of meta-skills such as "exercising effective 
learning control" (Wilson, et al., 1995). 

A final development issue involves the structuring of tasks, and the giving and receiving of feedback 
between participants. Riel's (1990) work on AT&T Learning Circles demonstrates one way of structuring these 
projects that is worth considering; each group follows a similar schedule ad guidelines for developing and sharing 
their work. 

5. Implementation (participants produce original work, share process information) 

During the implementation phase, learners work with each other and mentors to produce original work in 
teams. They are guided by the plans set forth in the previous stage. Increased interactions across the system may or 
may not occur. It is worth noting that while this is an "online" model, much of the work may take place offline. It 
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is difficult for developers or facilitators to keep track what is happening in distant locations. Regular updates should 
be provided from the sites. 

Additionally, efforts must be made to encourage participants to include "process information" in their 
updates — problems encountered, solutions attempted, lessons learned, changed plans, and so on. This is perhaps the 
most difficult problem that designers face, creating the expectation for reflection in and on action (Schon, 1983) both 
for the benefit of the group and the individual. Spitzer, et al. (1994) describe the process of "fostering "reflective 
dialogue" including different roles and strategies for facilitators. 

If participants do not document and share the process of implementation, opportunities for reflection, 
dialogue and feedback may be lost, and the end products themselves become less meaningful. It is necessary for the 
developers to allow learners and teachers to take control of and guide their own learning, while still offering support 
and encouraging interactions with the larger group. Having an audience itself can be motivating. "Students enjoyed 
working for and reading the work of other students even when they knew little about them” (Riel and Levin, 1990), 
and the receiving feedback can be beneficial, "The most powerful and effective source of feedback is another person" 
(Johnson and Johnson, 1996, p.324). In some, communication during project development and implementation 
offers important opportunities for learning — interactions with peers, mentors, facilitators, and SMEs that are often 
not available to classroom projects. 

6. Evaluation (assessing published work and providing feedback, evaluating the system) 

When one is finished with a project, an important part of the model is to share this product with others and 
request feedback (Rogers, et al., 1990). It is necessary to publish the work that took place, not only to help assess 
the project and its outcomes, but to facilitate learning which happens when learners "explain what they are doing and 
why" (Goldberg and Richards, 1995, p. 6). Presentation of work should reference the original goals, as well as what 
happened along the way. Given that this presentation may be asynchronous, e.g., provided to others via a Web 
page, it should include overview that describes the essential elements of the project for anyone who might serve as a 
reviewer, including the basics — who, why, where, when, and how.. .and what type of feedback is sought! It may be 
useful for designers to provide a template for introducing the project, a consistent structure, e.g., creating a sample 
web page that each group can modify to introduce their projectsA 

Formative assessment should happen throughout the development and implementation process - developers 
should be in contact with participants or facilitators on a regular basis. Rowland (1995) suggests that in learning 
communities feedback and modification is a daily process (Rowland, 1995, p. 60). 

Finally, the work can be summatively evaluated based on a number of criteria. Riel and Harasim (1994) 
discuss analysis of network design and structure, as well as social interactions, and individual learning outcomes. In 
terms of the latter, it is still necessary to work on the development of "technology-based strategies for enhancing 
assessment and the construction of portfolios" (Lin, et al., p. 60), and it may be advisable to assess some student 
learning outcomes through complex pre-post testing. 

To some extent a successful design can be inferred if it fosters sustained use by participants, as well as 
through systematic observation and feedback (p. 14-15). An important social interaction issue concerns the extent to 
which participants feel comfortable taking "risks" in sharing their ideas. "A free exchange of ideas, opinions, and 
feelings is the lifeblood of collaborative learning (McKinley, 1983, p. 16). It is also necessary to consider progress 
toward goals: 

Teachers should become innovators in terms of their own methods and their support for student 
collaboration, the community should increasingly perceive students as authentic contributors of 
knowledge, and the participants should be able to reflect on their own experience, decisions, and 
progress. Ultimately, what students create should be valued not just by the students and teachers, 
but by members of the broader community. Artifacts of learning should be exploited as such, not 
only by individual teachers, but by the community of practice (Will Duggan, personal 
communication, 3/23/96). 

Finally one should consider the system itself - its sustainability, transferability, costs and benefits 
(Collins, personal communication, 10/29/95). By showing these outcomes, learning communities can "be 
accountable to larger constituencies" (Lin, et al., p. 60). Further discussion on the evaluation of collaborative online 
projects, including methods for the collection and analysis of data is available (Kozma and Quellmalz, 1996; Ravitz, 
1997a; Ravitz, 1997b; Riel and Harasim, 1994). 
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7. Dissemination (sharing information on process, outcomes, and useful products) 

Finally, is appropriate to expect findings to from this work to be shared, and for successful efforts to be 
disseminated through the marketplace, or through the educational system. The "moral purpose" (Fullan, 1993, p. 4) 
must be remembered - to bring about improvements in the way young people are taught and learn. The reason for 
dissemination is so others can try to replicate and improve on previous work. Much discussion now centers on the 
profit motive in education which many see as a threat. It is worth noting that two of the models discussed here were 
developed with corporate financing and are available, in full, on the Web for public viewing. Regardless of the 
reasons why this work is on the Web, the opportunity should not be lost to view it, and to attempt to improve upon 
it. A larger issue concerns the ability of this work to be reflective itself, demonstrating lessons learned by reporting 
problems encountered, obstacles and how they were overcome. There must be some effort to develop the research 
focus of these development efforts and share evaluation results so that more can be learned from them. 

Analysis of the proposed Model 

The model includes classroom-level implementation, production of artifacts of learning, and an active 
network of people — a sustained learning community comprised of people, processes, and products (Table 1) -- that is 
actually a system in itself (Gustafson and Powell, 1991, p. 3). This model attempts to account for the major tasks 
(Table 2) involved in developing an online collaborative learning environment. 

Using the framework provided by Edmonds, Branch & Mukheijee (1990), it can best be understood as a 
descriptive model, derived mostly from procedural, as opposed to causal, knowledge. It can only be considered a 
"soft-system" with loosely determined interaction between components. Implementation requires a high level of 
expertise — as one must manage the complexity of the interactions, provide facilitation, build relationships, and 
support group work in the classroom. The work extends to multiple levels, but ultimately is intended for use at the 
classroom level. 

Using the framework provided by Gustafson and Powell (1991), one must note that this model requires a 
high commitment of resources and a team-based development process. The model requires careful selection and 
development of materials and assessment of participants’ needs. Tryout and revision of this approach should occur 
over the course of many iterations with opportunities for distribution and dissemination built in as an inherent 
advantage of using the Internet as a medium. The model has generalizable characteristics, and replication of the 
process is encouraged for research purposes both within and beyond the K- 12 contexts discussed here. 

The Model Components (People, Process, Product, and Participation) 

Key components of the model include people, process, and product, as well as participation. Instead of the 
learner working alone in "the world's greatest library", this model is based on connections between people from 
different walks of life. It attempts to create a process for the sharing of work in progress, and to provide products of 
learning that can be reviewed and discussed. 



People: A model that 

breaks down the isolation 
of students and teachers 
from the outside world. 



Process: Building a 

community of learners 
who can share their 
work in progress 



Product: Participants 

develop a number of 
products throughout 
the collaboration process 



* teachers 

* students 

* facilitators 

* mentors 

* online resources 

* local community groups 

* subject matter experts 

* information providers 



* identifying a theme 

* identifying resources & 
mentors 

* creating a delivery mechanism 

* receiving commitment 

from participants 

* facilitating interactions 

* supporting project 
development 

* publishing 

* structuring feedback 



* groups established 

* mentor identified 

* email messages sent 

* projects defined 

* multimedia presentations 
delivered 

* student online portfolios 

* feedback from peers/experts 

& teachers 

* evaluation data on people, 
process and product 



Table 1. Examples of People, Process and Product 

The Fourth P: Participation 

A key "formative" issue to be addressed throughout the development of such a project is to make sure that 
people are involved and committed to the process. As Riel and Levin (1990) warn, "Too many networking 
communities have fallen silent, as electronic ghost towns .... a quick introduction at an electronic cocktail party 
which fails to lead to any further exchange" (p. 145-6). Riel’s (1990) work for the AT&T Learning Circles draws on 
this research by providing many check points where participants confirm their commitment to the process. While 
this model is not as structured, it does provide formative questions that developers should ask themselves at each 
stage: 



Stage of Development 


Key Activities 


Evaluation Issues 


Management 


Development of coordinated team 


Is the team responsive and 
demonstrating shared understandings? 


Front End Analysis 


Identify topic(s) of interest 
Develop or find relevant resources 
Create relationships with people and 
groups who can support projects. 


Is there sufficient interest? Are 
resources and personnel available to 
support inquiry? 


Design 


Provide tools for communications. 
Facilitate sharing of interests. 
Matchmaking with resources. 
Form groups across distances. 


Are people taking advantage of 
response opportunities and sharing 
information about themselves and 
their interests? Are all participants 
making connections? 


Development 


Groups develop projects with 
support of mentors and feedback 
Toe -> Wade -> Dive 


Are people sharing ideas and plans, 
and receiving feedback during 
development? Are mentors involved? 
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Implementation 


Support teachers and projects 
Publish updates 


Are people providing updates on a 
regular basis? Are they reflecting on 
their experiences and discussing the 
process? 


Evaluation 


Publish completed work 
Give and receive feedback 


Is there evidence that learning 
occurred? Is background information 
provided to guide feedback? Is 
constructive feedback received, and 
responded to? 


Disseminate/Replicate 


Developers share process 
information and products. 


Is the process of development 
documented enough to allow others 
to replicate? Are actual results 
demonstrated and discussed? Are 
difficulties and research issues raised? 



Table 2: Partial summary of activities and formative checkpoints during stages of development 



Conclusion ^ 

There are drawbacks and risks to using a model like this. It is expensive, time consuming, and requires 
technological sophistication that many schools and organizations still lack (Heaviside, et al., 1997). It also requires 
significant changes in educational practices that may face considerable resistance. "Poorly implemented, the 
redefinition and blurring of roles can lead to chaos and confusion" (Wilson, et al, 1995). However, it is likely that 
only by developing and making explicit the models used to create such projects will practice be better understood and 
improved. 

The projects presented here are essentially "non-ISD" examples. Although the author tried to present the 
work in light of an ISD framework, this is largely post-hoc. The author has attempted to argue for a more ISD- 
based approach to such work by placing it within such a context. 

The best work is likely yet to come as the power of Internet-driven educational applications increases 
(Guzdial and Weingarten, 1996) and the extent of access and use in schools grows (Heaviside, et al., 1997). 
Unfortunately, many of the most advanced applications of the Internet are taking place outside the realm of ISD 
research. Is this an acceptable situation? What does it mean if the "best" models are coming from outside of ISD? 

This paper may not have satisfied allegiant practitioners that principles of ISD have been followed in this 
model, or that the strengths of traditional ISD have been fairly represented. Nonetheless, this paper makes an effort 
to place the development of online learning communities into a framework for future consideration, one which draws 
on the strengths of both research communities. This paper has attempted to show that further dialogue is necessary - 
- ISD has something important to offer to this type of work, and can learn from others as well. 



* The online version of this paper includes a series of matrices showing the interaction of these components (Table 
1). It also includes pointers to examples and online references, and initial attempts to create a graphic depiction of 
the model. It is avilable at - http://idde.syr.edu/HTML/Ravitz/ide_model.html. 
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Abstract 

This paper discusses professional ethical concerns for individuals conducting research in electronic 
collaborative environments. In order to explore this issue, the author reviewed existing professional codes of ethics 
and engaged respected figures in the field of educational technology in conversations about ethics, professional 
communication, and new technologies. Findings are presented concerning existing codes of conduct and challenges 
posed by new technologies. 

Introduction 

This paper is the result of a unique opportunity to examine ethical issues as part of a graduate-level class on 
research methods. If graduate students are not exhorted to consider ethical behavior as a part of their training, it is 
doubtful that this will be considered in their professional careers. Thus, students should have a chance to consider 
professional ethics as part of their training. 

Background 

The question of professional ethics is one that has been addressed by numerous professional organizations. 
For an overview, please refer to a meta-analysis of professional codes of ethics provided by Kultgen (1988). Only 
some of the professional ethics codes pertain directly to scholarly communications, notably — 

1) ensuring professional competence 

2) protecting the client, and 

3) protecting the profession. 

Each of these relate to electronic communications because new channels are available for interaction via the 
Internet that open up possibilities for representing and misrepresenting individual and organizational characteristics. 
Electronic interactions can change the relationship between professional practitioners and clients. Can existing codes 
of professional ethics manage this change, or do they need to be updated? The Code of Ethics for AECT offers some 
guidance in at least three areas relevant to the Internet: 

1) to protect privacy and personal integrity, and to 

2) distinguish between personal and organizational views, and, 

3) to give credit where credit is due. 

Findings 

All of these are issues for the electronic age. At the same time that one benefits from using e-mail as a 
means of communication, representing the ideas of individuals and organizations becomes a sensitive issue. While 
organizations may have once controlled the proliferation of information, in the electronic age each individual 
becomes an emissary, potentially influencing the views of others toward an organization. It is common to see 
disclaimers at the bottom of e-mail messages stating that the views expressed belong to the individual, not the 
individual's organizational affiliation. However, this line can become blurred and coordination between co-workers 
within an organization may become a thorny issue as messages are communicated across the Internet. It is unclear 
whether policies governing employee use of the Internet can adequately address this issue without stifling external 
communications. It seems safest to assume that an individual is speaking for himself, unless it is explicitly stated 
that an organizational position is being offered. 

Another issue involves protecting the client and the profession against misrepresentation by individuals of 
their professional competence or organizational affiliation. Even the "reality" of who an individual is, or who said 
what, can be obscured via electronic communications as individuals become more like disembodied beings. Clearly 
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in the context of professional communications representing accurately who one is in a communication, including 
one's institution and credentials, is essential. The ability to know who one is talking to, and properly identifying 
who said what, would be a requirement for ethical and professional behavior. In many cases, people may unwittingly 
or intentionally wrongly attribute the source of a statement. 

Furthermore, one hopes that individuals maintain ownership of their ideas when these are shared online. 
Schrum (1995) writes that people do have ownership of their communications, and should be able to modify them if 
they want — they should be informed if someone is researching their communications, and for what purposes. 

Perhaps the most important finding is that there is an inherent conflict between giving credit, and protecting 
privacy. It is impossible to do both at the same time. This becomes an issue as personal communication 
proliferates and becomes easily reproducible. Under some circumstances it may be ethical to cite someone by name, 
while under different circumstances it is necessary to protect their identity. For example, it is necessary to give 
credit to the ideas of others, but perhaps this is only acceptable with their permission. The generally accepted 
practice seems to be to ask permission to cite e-mail communications in a paper. However, what about reproducing 
group discussions? Dr. Lloyd Rieber, from University of Georgia, is owner of a listserv called ITForum. While 
information from this listserv could be broadly useful if disseminated more widely, making this information easily 
accessible might violate the spirit of the original discussions. "Would we need to get permission from all of the 
people who participated (potentially hundreds from all over the world)? (Rieber, personal communication, 1 1/8/96). 
Because the postings were already available via the Internet, ITForum feels it acceptable to make them more easily 
available via the World Wide Web, but they are making efforts to protect authors who might not have anticipated 
their writings being shared more widely, particularly if the material was sensitive or personal (Rieber, et al., 1997). 

Organizations must develop their own policies, and ask if existing ethical codes address the use of new 
"interactive" technologies. The null hypothesis would be that existing intellectual property guidelines are sufficient. 
Schrum, who has written extensively on this issue tends to agree with the latter notion: "We are not really changing 
our views of intellectual property, but rather some individuals feel that they do not need to abide by them anymore. 
I don't mean to sound as if things are rampant, but there is a definite and growing view that anything on the web or 
in digital form is fine to use/modify/duplicate" (Schrum, personal communication, 11/9/96). 

Conclusion 

In the end, the profession is not responsible for individual behavior; individuals are. The Chair of the 
Professional Ethics Committee for AECT, Dr. Andrew Yeaman, has indicated that ethical issues exist within a larger 
framework within society. Professional ethics are only one way of controlling people based on the status of the 
profession in society. Appeals also must be made to individuals, groups, and society at large. 

Professional ethics reflect the ethical concerns in society at large. There are larger ethical concerns 
involving the social changes taking place as a result of new technologies, issues that go beyond professional practice 
alone to the larger social and ethical effects of the computer age (Kizza, 1996). For example, while some have 
argued that electronic communications can foster a Jeffersonian democracy, Yeaman points out that no one ever asked 
whether anyone wanted this! Such an irony for Jeffersonian idealists! 

Society has a lot to grapple with, and so do scholars who seek to benefit from this medium in the context 
of professional and ethical practice. The literature on professional ethics seems like a logical starting place, but it 
may be necessary to move beyond current thinking to meet the challenges of new interactive environments. 
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The domain of instructional design has always been rooted in research and theory, initially originating from 
those of other disciplines. As the instructional design field matures, the bulk of current design research is now being 
conducted within our own intellectual framework and addresses our own questions more directly. Moreover, most 
feel that the Design Domain is the most theoretically mature domain of the field of Instructional Technology. In 
spite of this, some are arguing that not only the validity of the design theory base is questionable, but more 
basically they are questioning whether we have formal theory at all. Some note the preponderance of “theory” 
suggested originally as an application of foundational research in other disciplines, but still not validated with 
design-related research. As such, the field is often in a position of accepting hypothesized theory as established 
theory. This same dilemma exists with respect to many of the instructional design procedural models. The models 
have been devised based upon practical experience and a synthesis and an application of other foundational theory. 
But many still lack empirical support and validation in their own right (Gustafson, 1991). 

At the heart of these discussions is the lack of fundamental agreement as to the nature and function of 
instructional design theory. Should design theory be descriptive, prescriptive, or predictive? What foci are 
appropriate for design theory? The design process? The instructional process? Factors that impact instruction and 
learning? Factors that impact designer decision making? It is interesting that these conflicts pertain to instructional 
design, supposedly the most theoretically “sophisticated” facet of our field. 

The answers to these questions depend not so much on one’s position as to what is good theory or bad, nor 
to the relative merits of positivism or post- Aristotelian thinking (to use the Jonassen, et.al., 1997 dichotomy). To 
a greater extent the discussions seem to hinge upon one’s basic definitions of “theory” and “instructional design” 
themselves. This paper will address these two notions, as well as the connections between the two. Fundamentally, 
I am arguing for a broader conceptualization of both instructional design and of instructional design theory than is 
often typical. I am suggesting that this broader view of both theory and discipline is necessary to establish a truly 
comprehensive and functional theory base. 

The Definition of “Theory” 

Theory has been defined in its most traditional sense as “a set of related propositions that attempts to 
explain, and sometimes predict, a set of events” (Hoover, 1992, p.66). An alternative, but not totally unrelated, 
definition suggests a theory is constituted in terms of “two or more variables linked by rule and a set of limiting 
conditions” (Meehan, 1994, p. 115). Others have identified various types and levels of theory which vary in terms 
of format and formality. (See Richey, 1986, for a more detailed discussion.) Theories come in many forms, and 
have varying degrees of robustness. They can be formal systems of laws and propositions. They can be narrative 
explanations and predictions. They can be models which are either verbal, visual, or mathematical. 

Systems of Laws. Theories formally constructed as systems of law are the most traditional theory format, 
and at the same time the rarest theory form in Instructional Technology. However, the term “theory”, used in this 
sense, can take many forms short of being a full-fledged system of laws. It can take the form of an organization and 
summary of existing knowledge (Littlejohn, 1978). It can take the form of a hypothesis (even an initial and 
unsubstantiated hypothesis), or the form of a set of related propositions (Snow, 1973). These “theories” are built 
around suggested relationships among variables which have been established through research. Systems of laws have 
goals of both describing and, ultimately, predicting events. 

Formal theoretical systems to a great extent are missing from the design literature. This reflects current 
trends in theory construction as well as the lack of support for replicated, basic research in our field. Nonetheless, 
there have been some attempts (Merrill, 1994; Richey, 1992). However, one can characterize much design-related 
research as attempts to describe key variables and their relationships to each other. These studies are, by and large, 
attempting to establish fact. Even though frequently such research is not designed as a step in systematic theory 
building, it can be synthesized and used to develop hypotheses. 



Narrative Explanations. Theory can also take the form of a narrative description of events. While systems 
of laws tend to be based upon quantitative research, narrative explanations (in this case, explanations of the critical 
design processes and their outcomes) are more often based upon qualitative research. This type of theory has been 
described as being the “mirrors of man” and is developed so that we can see ourselves better (Kluckhohn, as noted in 
Diesing, 1991). These narrative explanations create a knowledge base by providing “descriptions, from the inside, of 
a way of life, community, person, belief system, or scientific community’s beliefs (Diesing, 1991, p. 325)” and 
serve as models of dynamic processes of a field. 

There is a new body of research in our field relating to designer decision-making and problem solving. This 
research has the potential of developing into theory of the type as described by Diesing. Much of this research is 
qualitative (Nelson, 1990; Rowland, 1992), and its primary goal is to understand the process of instructional design 
and the nature of design thinking. This research describes and explains factors such as the instructional design task 
environment (Goel & Pirolli, 1988; Kerr, 1983), and the cognitive process of instructional design (Akin, Chen, 
Dave, & Pithavadian, 1986). This body of literature is leading to the development of new theoretical constructs (e.g. 
designer thinking, design task environment), and can be viewed as elements of narrative theory construction. 

Postmodern criticism or theorizing can also “fit” into this category of theory. Interpretation from unique 
perspectives without concern for generalizability, or reflection on the essence of meaning can result in a narrative 
description of complex issues (see Wilson, 1997). 

Models. The third form of theory relates to models of dynamic processes and procedures. Such models can 
be conceptual or procedural in nature. Even though there is a historical precedent for such theory to arise from basic 
research, it is not essential. Theories can be rooted in applied research conducted in real-life settings. 

For many, the design procedural models form the core of design theory. Dick (1997) cites the role of 
models as “representations of theory”, and notes that they serve as techniques for summarizing the research and 
procedures of our field, and of visualizing a succession of “if-then” statements. They can be easily supported by both 
evaluation research and developmental research. 

Most design procedural models are variations of the traditional Instructional Systems Design (ISD) models 
(see Dick and Carey, 1996). However, there are also design models that provide a structure for instructional strategy 
selection and sequencing as Gagne’s Events of Instruction (Gagne, 1985) or Reigeluth’s (1983) Elaboration Theory. 
We also have non-procedural design models that shape our thinking as well. These are conceptual models. Seels 
(1997) describes a number of these models in her analysis of design-related taxonomies, including Dale’s Cone of 
Experiences, the learning task classifications, and Clark’s Taxonomy of Media Attributes. 

The Scope of Instructional Design 

There have been a number of different definitions of instructional design with most highlighting design as a 
planning process (Briggs, 1977; Richey, 1986; Seels and Richey, 1994). However, there are those who emphasize 
only the planning in terms of instruction and instructional methods (Reigeluth, 1983), as opposed to the more 
comprehensive activity that also includes, for example, content selection, planning for assessment and evaluation, 
and instructional management. Clearly, the parameters established for design itself impact the nature and scope of 
the corresponding design theory. Much of this confusion may stem from the early history of instructional design 
when the more common term used was “instructional science.”. 

I have previously separated these two points-of-view (Richey, 1993, 1995) by distinguishing between 
macro-design models and theory (i.e. ISD models describing the entire design project) and micro-design models and 
theory (i.e. lesson design models). While I see these all as instructional design models, the distinction does reflect 
design’s different intellectual roots and foundations. Macro-design primarily reflects the influence of general 
systems, curriculum, and management theory. On the other hand, micro-design reflects the primary influence of 
psychological, instructional, communications and audio-visual theory. 

Ragan and Smith (1996) have somewhat avoided the design definition dilemma by using the more precise 
terms “conditions-based theory” and “conditions-based instructional design models”. Such terminology clearly 
reflects the influence of Robert Gagne, as well as the psychological and instructional theory foundations of design. 
It also fairly precisely categorizes a large segment of design theory. 

It may be that there are other design theory genres that also provide alternative views of instructional 
design. For example, there are portrayals of design as a problem solving/decision-making/reflection activity, or as a 
scientific engineering activity, or as a performance improvement activity. There are also genres of design theory that 
reflect varying philosophical points of view, such as constructivism or post-modernism. 
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Thus, the problem may not be so much one of conflicting definitions of instructional design, but rather one 
of conflicting conceptions of the design knowledge base. It may be a classic case of assuming the part one 
understands or gives credence to is actually the whole. It seems that progress in this field of study is dependent to a 
great extent upon accommodating a comprehensive view of instructional design. This more inclusive scope not only 
lends credence to theory addressing a wider range of topics, but also facilitates theory constructed in alternative 
formats and alternative intellectual orientations as well. 

Instructional Design Theory: Dream or Reality? 

The question then is whether that which we call design theory is truly theory and whether it truly represents 
the field of instructional design. Typically, theory is judged in terms of its accuracy, validity, and its utility. The 
additional element introduced here pertains to the comprehensiveness of the body of design theory as a whole. 
Topical breadth is a function of both the prevailing views of the nature of instructional design as well as the 
prominent issues that are currently attracting attention (Richey, 1997). Breadth of theoretical format is to some 
extent is a function of the same factors. However, there has been a narrowly defined view of theory in the history of 
instructional design. This tradition tends to recognize theory primarily in terms of either: 1) ISD models or 2) 
specifications of rules for instructional strategy selection and sequencing. In spite of the many contributions such 
theory has made to the field, the continuation of this somewhat narrow intellectual framework can pose serious 
problems for the field. Accuracy can be doubted because of a perceived lack of relevance. Relevance can be sacrificed 
because of topical provincialism. Validity can be threatened by methodological stagnation. Ultimately, such events 
can result in the utility of our instructional design theories being generally ignored by all except the theorists 
themselves. 
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Stories of our Teaching: Educational Technology in Context 



Rhonda S. Robinson 
Northern Illinois University 



Introduction to the Panel 

Teaching is more than the sum of its parts. Teaching in any field, at any level, involves many human 
aspects of communication, thought, and caring. And teaching at the university level, in educational technology, has 
been too little discussed and investigated. The aspects of caring, of communication, and of dialogue that are 
embedded in our teaching have for the most part been ignored in our reports of program practices and curricular 
innovations. We somehow find it more important to discuss our learning theories, our authentic tasks and 
apprenticeships, or our assessment procedures. 

But many of us find that the stories of our teaching are becoming insistent; they want to be told, retold, 
shared, and discussed. The dialogue that we should be having involves not just our theories or our research, but our 
methods of teaching. This panel presentation and series of papers begins to look at an emerging area of educational 
technology teaching, from a narrative perspective. 

The idea of narrative in education is growing more popular and vital. Witherell and Noddings (1991) have 
presented a collection of essays which explore this narrative from a variety of content perspectives. We plan to 
follow their example to provide narrative explorations of our teaching. “The use of narrative and dialogue can serve 
as a model for teaching and learning across the boundaries of disciplines, professions, and cultures” (p.2). 
Acceptance of narrative and story has grown to include many other disciplines. For example, the NCTE 1996 annual 
conference theme was “Honoring All Our Stories”. The program guide introduced the theme by saying “listening to 
one anothers’ stories is a way to honor our differences and discover our commonalties. Through such sharing we can 
consider how our individual stories weave with those different from our own to form the fabric of our community.” 
(Avery, p. 3). Many of the presentations at that conference focused on using narrative to help students learn; one 
such panel was entitled, “Telling our Stories Out of School”; another was ‘“How Stories Define Us”. 

The teller or writer of a story adds meaning to that story, “meaning embedded in his or her culture, 
language, gender, beliefs, and life history. This embeddedness lies at the core of the teaching-learning experience... ” 
(p. 3). Bruner (1985, cited in Witherell and Noddings, 1991, p. 3) called stories a part of the way we understand 
human experience, as important as the paradigmatic or logico-scientific mode of knowing. Teaching involves both 
ways of knowing; “it calls on both narrative and analytic ways of knowing... it asks us to address the moral and 
aesthetic as well as the practical aspects of everyday experience” (p.9). It is these practical aspects and everyday 
experiences which we feel need to be shared and discussed. 

Teaching is more than pedagogy; it is personal and individual, and helps the teacher as well as the student, 
since “...stories help us find our place in the world; caring, respectful dialogue among all those engaged in 
educational settings...” helps us come “to understand ourselves, others, and the possibilities life holds for us” • 
(Witherell and Noddings, 1991, p. 10). 

Teaching is, among other things, a discursive and interpretive practice, just as the writing of 
autobiography is. Teaching is textuial. When we teach, we tell stories about the world. Some 
stories are scientific, some historical, some philosophical, some literary and so on. Educational 
theories are stories about how teaching and learning work, about who does what to whom and for 
what purposes; and, most particularly, educational theories are stories about the kind of world we 
want to live in and about what we should do to make that world. Stories... help us to find our 
place in the world(Pagano, p. 197). 

Each paper in this group introduces the author/professor who has designed and taught a course in educational 
technology from a socio-cultural or alternative perspective. Our courses are for undergraduates and graduate students, 
on technology, design, and research topics. Each presenter has developed their own theoretical stance, has shared 
their thoughts and feelings as well as content and structure throughout a class, and has had the course evaluated and 
revised through student input over the semesters. The courses include a doctoral seminar in research topics, an 




undergraduate media and technology in teaching course, an alternative instructional design and development class, and 
the issue of Conceptual Analysis developed within two different educational technology courses. 

This symposium shares the stories of these courses with participants, asking them to help develop the 
texts, be involved in the stories, and share the dialogue with the panel. These papers engage their readers in the 
ideas of narrative as a part of teaching, and encourage them to consider alternate approaches to teaching and learning 
that accompany the course content, background and methods presented. 



You Have a T-Rex Here? 

Rhonda S. Robinson 
Northern Illinois University 



Introduction 

In completing a class last semester, students were doing presentations as part of their assignment. Two 
students were giving a train-the trainer session on the Internet, and as part of their presentation they had furnished 
little toys and give-aways for the participants. They had selected one especially for me, and gave it to me with 
smiles or smirks on their faces. It was this dinosaur... a symbol I took seriously to represent something about 
myself. I then told the class the story about a colleague’s saying to me that he and I were like dinosaurs, who were 
going to die in the mud, if we didn’t become more avid technology users. 

In that same class, one night I was planning to show a movie entitled Animation Pie , which shows a late 
1970’s film production in the schools project and features several animated films and their young filmmakers 
working. One of the class members turned and saw that I was using a 16mm film projector, and exclaimed quite 
loudly, “Oh my word, how do you even know how to work one of those? I’ve never even seen one before!” After 
we all had a good laugh, I realized that the gulf between myself and my students was ever widening; but I planned to 
stand my ground. How did I learn to work one of those, indeed. ..that’s what got me into our field in the first place. 

Personal Background 

I started out as a English major, like Randy Nichols (see related paper), and also learned to explicate text as 
a part of my academic preparation. After I student-taught at the middle school level, all administrators who 
interviewed me thought I’d be “perfect for their junior high,” even though I had a master’s degree in American 
literature and wanted to teach Twain. So I ended up teaching 7 th and 8 lh grade language arts to kids who didn’t care 
about Twain or Hawthorn or poetry or even sentence structure... but they did like media. Our language arts team 
then created a film club, produced the school newspaper, and taught open reel video production in our classes, 
including a long unit on television and news, advertising, and American humor as revealed by television. 

After five years of that, I decided for personal reasons to go back to school, and investigated the 
Communication Arts program at Wisconsin, my alma mater, to see if I could earn a second master’s degree in media. 
After I was admitted there, I was blessed by an incredible piece of luck — I took the time to try to meet Ann DeVaney 
and interview with her regarding the Educational. Communications program. . .and knew I’d found my home. I spent 
four years with her and was tremendously gifted to work with her as a TA, teaching classes in the program. Two of 
many defining events were my taking a film analysis class and then minoring in film study , something that had 
little precedent in education ( for someone to have a doctoral minor in Communications / Liberal Arts and a major in 
Education). The second event was being encouraged by Ann to study research methods and statistics with an 
educational anthropologist and sociologists rather than with the Educational Psychology statistics faculty. I 
conducted what I back then timidly called descriptive research, but I actually did one of the first qualitative studies in 
our program. Impacting all of these and other experiences was the influence of working with Ann, whose interest in 
critical theory, and textual analysis pervaded all our work. Her openness to alternate viewpoints and her 
encouragement regarding qualitative research was the pivotal influence in my development. She should be credited 
with having that impact on everyone in the field, not just her own program, as she published and spoke about these 
issues very early on. She also graduated a group of us, Streibel, Mclsaac, Considine, Chute, Muffaletto, Koetting, 
Knupfer,, who went on to continue growing and spreading the critical viewpoints in which we were fostered at UW. 
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Teaching Perspectives 

As I’ve developed as a professor at NIU, I have continued to bring the textual analysis and media emphasis 
of my background to the classes I design and teach. I have added critical theory readings to our doctoral seminars, I 
developed a course in visual literacy, and I encourage qualitative research for our students, teaching a course in 
naturalistic inquiry and infusing those ideas into all our seminar classes. The class I want to explain in this 
presentation is a doctoral seminar, required of all students, that focuses on the research problems in our field, and 
allows students to develop a literature review and a preliminary proposal on their chosen topic. 

The class also includes reading about the “big” issues and problems in our field, and includes an assignment 
for them to read and present to class the work of some of the more critical thinkers in the field. (At an earlier 1995 
presentation at this conference, I presented on this class and shared the Appended reading list). The responsibility I 
have taken on is to move students through several stages, from disinterest, to discomfort, to grudging admiration, to 
acceptance of alternate or critical viewpoints. It is a sometimes painful process of having students rebel, complain, 
and dodge ideas, and I am able to report only partial success with my model However, I can share some stories from 
last semester. 

Early in the class, people were reluctant to read from the assigned list. They wanted to chose Gates or an 
advanced design model text, not Paradigms Regained. The students start the class as what I consider typical 
educational technology students — they are technology directors, learning resource deans, administrators of distance 
education systems, help desk managers, and so on. The ideas inherent in the readings I assign are often very new to 
them. They are comfortable reading Bill Gates, but not Chet Bowers. They love sharing the ideas from Jonessen, 
but not from Muffaletto. But those that stuck with the suggested readings persevered, and presented these new ideas 
to the class. 

The assignment to give a presentation to the class took several forms. For instance, to summarize his 
presentation, one person presented a series of film clips which portrayed, from older science fiction films, the views 
our culture has produced which show the “dark side” of technology. As we watched Michael Rennie as an alien or 
Peter Sellers as Dr. Strangelove, we contemplated the truths hidden in that popular science fiction. 

Another student took us through a presentation obstensively about Bill Gates’ ideas, but used several 
different instructional strategies which turned out to be those following Gardener’s Multiple Intelligences. So while 
he shared more conventional ideas, he used the medium to be the message, and helped us feel our multiple ways of 
learning, the real point of the presentation. 

A third student who spends his days crunching data for the social science research center presented on 
qualitative data analysis, and helped us understand the difference between the methods of conducting research using 
his own examples. 

A fourth class member used Saul Alinsky’s early work to help us see the issues of social action, 
considering it a type of diffusion of innovation from a new perspective. He encouraged the class to consider 
innovation as an answer to the question, “What can I do to make a difference?” He addressed the training needs of 
this changing society, and considered the problems of today’s constant need to learn. He also looked at case studies 
as a way to frame research problems. “I still feel that humans are story tellers (narrative), and this is how we convey 
and store information, and make meaning of our lives. Narratives also shape our relative environments, and give 
clear understanding from a realistic standpoint without the manipulation of numbers (considering you can work 
numbers any way that you want)” he told us at the end of the class. 

And finally, one international student (and these presentations were sequential, building to this one) very 
cleanly and succinctly summarized post-modernism and critical theory, and summarized for us the ideas towards 
which we had basically been evolving, putting the process into new perspective by presenting for us how we’d 
progressed throughout the semester to new understandings of these perspectives. 

About halfway though the semester, one student came in fairly angry about some of the e-mail discussion 
that had taken place during the week, and said to us, “But don’t you think it’s very interesting, I mean just so 
interesting , that we, instructional technologists who promote technology all day, are being asked to criticize 
technology and find its negative capabilities? I mean, isn’t that interesting!'' After he finished, I allowed a long 
pause and then asked, “Just what do you mean by interesting?” And from there, the discussion proceeded about 
discomfort and anger and facing new ideas. 

But by the end of the semester, my point of view and my purpose had been more than adequately addressed. 
Students really did understand that they may now ask more interesting, more critical questions than those they 
originally considered for their research. They moved off the “test” questions, the “how does it work” questions, the 





comparative questions, and began to understand the field’s need for the more critical questions regarding ethical 
effects, and the qualitative questions of how does this really work in the world of education? I felt successful in 
helping them grow to regard our field in a larger sense, in its impact on education and the world at large As I 
continue to work with these student this semester, I look forward to their research proposals and research questions. 
I hope that the reading we discussed last semester has allowed them to re-examine their topics and look for the more 
meaningful questions. 

I often still feel like that dinosaur I showed you earlier, though I think I am making progress towards new 
learning in our technical field. In doing this paper, I worked on a laptop computer, riding in a car, using a cellular 
phone.. .but I still had trouble seeing the cursor and using the stupid touch pad instead of a mouse. And I still don’t 
know how to create a great file management system so I don’t lose important documents. However, more 
importantly, the ideas that our presentations and papers represent ar e not dying in the mud; they are in fact growing 
and becoming more readily a part of the language of this field. The personal tone of these presentations may 
encourage us all to continue the dialogue and take our academic discourse to new levels of narrative truth. 

After all, as Crichton and Spielberg have shown us, dinosaurs may actually live again. 
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Print vs. Online Scholarly Publishing: Notes and reflections on the 

peer review process 

Martin Ryder 

Storage Technology Corporation 



The idea of publishing is a concept which dates back to the Fifteenth Century. The practice was enabled by 
a collection of several technologies merging together to forge the age of print. Printing effectively transformed 
society from an oral culture to a culture of literacy (Eisenstein, 1979; Ong, 1982). Today we encounter another 
collection of technologies which speak of major transformation, promising changes as profound as those which sired 
the modem age. The postmodern extension of what it means to publish is a fascinating issue which invites new and 
creative ideas unconstrained by traditional assumptions. This panel, by the questions it addresses, offers a forum for 
testing some of the major shifts in our thinking about the nature of publishing, and about our basic beliefs regarding 
the peer review process. 

Who owns the text? 

Print publication requires an outlay of capital for production and dissemination of each published volume. 
The expense is usually underwritten by a publisher who requires a return on the investment. Because the investment 
is not trivial, ownership is attached to the work. This secures the rights to any returned value in order to protect the 
investment. The institution of mass media is sustained on the notion that media objects can be owned and controlled. 

But that modern notion of ownership is suddenly threatened in cyberspace. Suddenly the role of publisher is 
trivial: I can “publish” your work by placing a mere pointer from within my own work. Or I can actually host your 
text on my server at a cost of pennies in disk storage. The modern publisher has little say in this postmodern arena, 
and there is a natural realignment of all other parties connected by the text. Ownership is now concentrated primarily 
between author and reader. The classification, “reader” includes all readers, whether editor, reviewer, student, librarian, 
web master, or any other patron who invests time and resources with the author’s work. 

The concept of ownership is a bit different here. Using the Using the Lockean notion of personal property 
(Locke, 1690), we acknowledge the author’s claim to ownership on the basis of personally invested labor. By the 
same Lockean token, we must acknowledge claims of ownership entailed with each act of reading: an investment in 
kind, if not in degree, with the author. From the standpoint of the text, the author relinquishes exclusive ownership 
the moment the work is made public. The value of a text is determined as much by the reader as it is by the author. 
Indeed, if there are no readers, what possible value could be claimed from the text? The economics of cyberspace 
allows this natural amalgamation of interests to exist between author and reader. 

Quality control: a summary or an emergent process? 

With print publications, the referee system is a summary process. Capital outlay is associated with each 
printing, and sound business practice requires an assurance of objective value and viability of a transcript. The 
author’s submission is evaluated against a set of specified and unspecified criteria. The submitted draft is either fit or 
unfit for publication. If the work generally lives up to expectations, the author is invited to revise any weak points 
and resubmit the article. Otherwise submissions are either accepted or rejected for publication. The draft that is 
ultimately accepted becomes the article that is published. Drafts prior to publication are rarely seen by the reader. 
Revisions beyond that point rarely merit the expense of republication. 

Not so with electronic text. There is no reason to withhold public access to an online article that is in 
process. The Net offers malleability where the printed page cannot. Continuous, open-ended revisions of text are 
feasible and even desirable in online environments. The recognition of this quality of the medium allows us to shift 
our conception of peer review from a summary process to an emergent process. Quality control need not be perceived 
primarily as an act of acceptance or rejection. The editor in an online environment can step down from the distant, 
anonymous, objective role as judge, and engage feely in acts of play and collaboration with the author. Quality 
control can be a process of mirroring, challenging, probing , validating, and encouraging an author throughout the 
developing cycle of peer review. 

So when does an emerging text become “publishable”? When does a draft become an article? A popular Zen 
koan has to do with the nature and process of being: When I say that “I exist”, who is this ‘T that exists? When did 
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“I” begin; when did “I” finally become “me”? The Zen master knows the absurdity of the question at the moment he 
offers it to his student. A text is an emergent phenomenon, no less than the human who conceived it. Imprisoned 
for centuries in the static medium of paper, text is suddenly free to develop, adapt and change without constraints. 
Within the domain of electronic substrate, we can consider the possibility of text as a living, organic structure, not a 
mere artifact. Like all other living phenomena, text can be understood for its emergent qualities, the ability to adapt 
and grow within the contexts from which it was conceived. Online text need not replicate printed text in order to 
become a legitimate medium for scholarship. Within electronic environments, a text can emerge modestly, then live 
on within the changing world that conceived it. 

A case for plastic texts 

This organic conception of text challenges our mythology about the sanctity of the written Word. Cultural 
tradition runs deep, imposing strong and tacit influences upon our present consciousness. A draft of an article 
commands very little homage compared to a fixed, permanent image of the text, even after the published article has 
become outdated by subsequent revelations. To illustrate this point, let me cite an example from my own workplace. 

StorageTek manufactures computer peripherals for large data processing environments. As part of the global 
trend toward online documentation, StorageTek’s manuals are going through the transition from hard copy to 
electronic form. Currently StorageTek provides both printed and electronic versions, the latter available to customers 
from the StorageTek web site. A recent engineering change in one of our products obsoleted some information 
contained in the information bulletin for another product. I called the matter to the attention of our technical writers, 
suggesting a minor revision to the online document. 

“Revise a released document?!” The writing group was totally unprepared for the suggestion. They cited any 
number of reasons, including ISO 9000 incantations, why it was a bad idea. The fact that users would benefit from 
the new information did not enter into their analysis. They were more concerned over the lack of fidelity to the 
printed Word. Knowing that the issue would be revisited many times, a compromise was offered. The “official” 
online text would never change. However, the writers would add a hyperlink from the official version to an updated 
copy. This “unofficial” draft can be corrected and updated dynamically as engineering changes and other conditions 
require. From the user’s standpoint, it is the draft version that holds promise for the most reliable information. 
Ironically, this version “under construction” holds subordinate status to the inaccurate, out-of-date, “official” version. 

The need for plasticity becomes more pronounced as knowledge becomes increasingly dynamic within a 
changing society. By placing text online, we can see the futility of preserving structure and content so carefully 
molded into a text by its author and so carefully scrutinized by an editorial body. Hypertexts are especially volatile 
because of their direct connections to external, uncontrolled sources. Texts which lend themselves to are perhaps the 
best suited inhabitants of the virtual library. When a monolith is fragmented into multiple “lexias” (Barthes/Landow) 
it allows external references to connect directly to contexts within the text. It also allows for dynamic, incremental 
text revisions which can be evaluated quickly within a responsive process of peer review. Any critical function 
related to the text (including the referee system) must acknowledge the organic, temporal, incremental, and 
deconstructive attributes of online text, the salient affordances which distinguish this medium from printed text. 

Studio/Gallery Model 

My intent in these musings is to kick off a discussion which allows us to come to grips with the task of 
“inventing” a viable, credible model for an online refereed journal. The challenge we embrace involves the 
“invention” of a referee model that makes use of the affordances inherited with a new medium while maintaining the 
credibility and intellectual rigor associated with traditional peer review. 

The plasticity of electronic text opens the way for interactivity as a means for quality control. This 
approach views text as an organic, dynamic phenomenon capable of adapting and changing within the context from 
which it was conceived. 

I have not yet considered the traditional conception of text-as-artifact, a legitimate paradigm that fits well 
within the medium of print. Being careful not to impose the print paradigm onto this new medium, it may 
reasonable for us to accept the fact that static representations can reside legitimately along side dynamic texts within 
electronic domains. Unlike print media, the World Wide Web can accommodate both very well. 

An essay is an artifact - a work of art - no less than a painting, or sculpture. Finished works of art are 
intended for public display, and it is a gallery that provides that function of public exposure for fine art. Like a 
gallery, a journal provides this function for textual artifacts. If the artist’s work meets thematic, technical, and 
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quality criteria then the work might be placed on display (though it is rare that a gallery fully discloses the actual 
criteria for inclusion). The focus in a gallery is on the artist’s work and its overall impact to visiting patrons. 

In the artist’s studio, the focus is on the process. Here the artist might display the emerging work to solicit 
comment and share ideas with interested students, peers and patrons. The process involves testing, validating and 
improving ideas or techniques that go into the work. Many works that can be found in a studio will never be seen in 
a gallery. Ideas which show promise might surface in later works, or new ideas might emerge out of specific 
reactions to earlier attempts. Works which never find their way to a gallery can still influence the state of the art. 

The focus in a gallery is the product. The work is on display to move, to provoke or to inspire the patrons 
who come to view it. The work is subject to criticism, not for purposes of improvement but to solicit reaction for 
its own sake. A gallery will strive for collections that provoke reaction in order to maintain a high level of public 
interest. 

It is conceivable that an electronic journal can provide both environments: an open studio and a showcase 
gallery for textual artifacts. In the studio, peer interaction allows for quality improvement, validation of ideas and 
techniques, and the generation of new ideas. The gallery is a showplace for ‘finished’ works. Gallery selections are 
likely made on the potential of a work to provoke public reaction and to maintain a high level of interest. It should 
not be necessary to limit displayed works to those that were developed in the open studio, and there is no guarantee 
that works developed there will find their way to the gallery. But artists who choose to work in the open studio will 
have the benefit peer commentary and feedback. 

Public reaction to displayed artifacts might find its way back to the open studio. This environment would 
allow for continued dialog in the form of peer commentary and criticism, advancing general perceptions of new and 
provocative ideas which were put on display. 

I believe this studio/gallery model might offer us the flexibility we need to implement open, interactive 
peer review, giving us the speed and diversity of opinion which it promises. At the same time we can reserve a 
degree of logocentric control over the actual publication to satisfy more traditional demands for academic quality and 
credibility. 
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Learning English Electronically: Formative Evaluation in ESL 

Software 



Heidi L. Schnackenberg 
Arizona State University 



This paper describes a formative evaluation of Learning English Electronically (LEE) conducted at Glendale 
Community College in Glendale , Arizona. A description of the program, as well as posttest , survey , and 
observation data will be provided during the proposed presentation. Implications of the compiled data will also be 
discussed. 

Description of Learning English Electronically 

Learning English Electronically (LEE) was developed with a grant from Apple approximately six and a half 
years ago. It is a computer software package designed for adult English as a Second Language (ESL) students 
enrolled in intermediate level community college ESL classes. It is intended to be used as a supplement to ESL 
classroom instruction. 

The program consists of 43 different lessons emphasizing various grammar concepts and accurate sentence 
structure. The 43 lessons are divided into 14 sections. The individual lessons take approximately 25 to 35 minutes 
to complete. Each section focuses on a different English grammar concept. Each of the grammar concepts has from 
one to five different individual lessons covering five different life content topics: employment, food, health, school, 
and transportation. At the present time, LEE is incomplete because all of the five topics are not included in every 
grammar concept. The program is designed for adult ESL community college students who are enrolled in 
intermediate ESL classes and have a moderate level of proficiency in the English language, but have difficulty in 
writing accurate and effective sentences. 

The courseware objectives vary from section to section, but remain the same for each lesson within a 
particular section. The various objectives deal with identifying grammar concepts correctly in context, using 
different grammar concepts correctly in sentences, listing grammar concepts according to category, and stating 
different forms of various grammar concepts. These are just a few general examples of the objectives contained in 
LEE Overall, there are 35objectives among the 14 sections. The program package consists of the an Instructor's 
Guide, Student Lesson Sheets, and 43 lesson disks with one main menu disk. (Hyper Card 2.0 application disks are 
also included in the package.) 

Evaluation Procedures 

The evaluation was designed to assess student and teacher attitudes and judgments about Learning English 
Electronically. Performance data in the form of test scores were gathered to assess the effectiveness of the program. 
Data were collected through (1) a survey administered to both students and teachers (2) personal interviews conducted 
with students and teachers (3) test scores submitted by students at the end of each module, and (4) observations. 

The surveys were completed anonymously by both the students and teachers. Responses to the choice 
items on the surveys were scored on a 1 (strongly agree) to 5 (strongly disagree) scale. For the open-ended items, 
similar responses to an item were categorized together and all responses are listed in the summary tables in this 
report. 

Personal interviews with both students and teachers were done on site at the Learning Center at Glendale 
Community College. Due to the fact that the students have difficulty with the English language, some questions 
from the survey were also asked in the personal interviews. The four topics covered in the student interviews were 
the method of use, areas of like and dislike, problems using the program, and miscellaneous comments. The four 
topics covered in the teacher interviews were the method of course implementation, program strengths and 
weaknesses, improvement suggestions, and whether or not the topics cover all the components of grammar they 
would like their students to know. 

Student performance data were also collected on site at the time of the personal interviews. As each student 
finished a module, he or she was asked to complete and print out the test at the end and submit it to the evaluators. 
Scores are given as the number correct out of the total number of questions for Sections A, B, and C of the test, A 
being the easiest section and C being the most difficult. There are also two questions in which the student can 
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construct original sentences and the teacher scores them after they are printed out. Each sentence is worth five 
points. This section will not be considered in the evaluation due to the difficulty in assigning point values 
consistently to the different components of the sentences. Parts A, B, and C will be scored on the percentage of 
questions answered correctly out of the total number of questions. 

Several other evaluation tasks were performed as part of the overall evaluation of the program. Reference 
manuals and several modules from the software itself were reviewed. Also, on site observational data were gathered 
as the evaluators were able to view the students actually using LEE and interacting with their instructors. 

Results 

The results are summarized below, first for the teachers, and then for the students. These are followed by 
performance test score data and observational data. 

Teachers Survey Responses 

Responses to the survey are summarized in Table 1 for the three teachers who completed the survey. The 
summary shows the number of participants selecting each response choice, the average score on each item on a scale 
of 1 (most favorable) to 5 (least favorable), and the overall mean score for the group. 

The overall mean rating by teachers for Learning English Electronically was 1.71, a very favorable rating 
indicating agreement to strong agreement with positive statements about the course. The highest rated items (1.33) 
were item 2 (The skills taught in the modules are valuable for my students), item 4 (The stories aid in the learning 
process), and item 7 (There are enough practice exercises). The lowest rated items (2.00) were item 1 (The modules 
are well organized), item 5 (The rules are clearly stated), item 6 (Instructions for the exercises are clear), and item 8 
(The modules are about the right length). 

The teachers thought the strengths of the program were that there was additional grammar practice available 
for the students if they took advantage of it, that the program was self-paced and non-threatening, that it has sound, 
and that the grammar topics were presented with content topics. The major weakness of the program was cited as the 
slow response of the computer in executing commands, while the potential improvements included making ESL 
students more aware of the program and increasing the speed of the computer. All three teachers stated that they used 
the program as a supplement to their classes and two said that they would use it again. One teacher commented that 
the program needs to be completed so that the students have access to more modules. 

Teacher Interviews 

The interviews with the two teachers dealt with method of course implementation, program strengths and 
weaknesses, and suggestions for improvements. The teachers were also asked if they felt that the 14 topics fully 
covered all the components of grammar that they wanted students to know. 

Both teachers said that they used the program strictly as a supplement to their normal course curricula and 
that students used it on their own time. One teacher stated that she would not feel comfortable requiring the students 
to use the program exclusively for class until the flaws in it were corrected. 

When asked about the strengths of the program, one teacher stated that doing a grammar lesson with several 
different topic choices (i.e. employment, food, etc.) while still learning the main focus of the module was interesting 
as well as effective for the students. Also, seeing grammar rules applied in the stories in context was good practice 
for the students. The other teacher stated that the program allowed students to work at their own pace and was a 
nonthreatening/nonembarrassing way for students to practice their English. 

Both teachers reported the fact that the students couldn't print out any screen of their choice, just end-of- 
module tests and some sentence writing exercises was a weakness of LEE. Other weaknesses cited were slowness in 
accessing information and some incorrect grammar and punctuation in the program. 

Both teachers reported that the 14 topics in the program fully covered all the components of grammar that 
their students needed to know. 

Recommendations for improvements in the program included increasing the speed in accessing information, 
improving the overall sound quality, and completing the content topics for all the modules. 

Student Survey Responses 

Table 2 shows the survey responses for the nine students who completed the survey. 
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The mean overall rating of the program by the students was 1.65, a very favorable rating. Item 10 (I liked 
using the program) and item 3 (the practice exercise helped me learn) received the most favorable ratings of 1.25 and 
1.38 respectively. Item 1 (The program was easy to use) and item 4 (The instructions were easy to understand) 
received the least favorable ratings of 2.11 for each item. One of the students thought the program was too short, 
while five thought it was about the right length. Seven of the students thought the modules were about right in ease 
or difficulty of use. Four students said that they used the modules on their own while four said that they used the 
modules because the teacher assigned them. 

Three students stated that the thing they liked least about the program was that it works too slowly, while 
one student stated that any screen they desired could not be printed. 

Recommendations for improvement included making the response time faster (2), including more exercises 
and tests (1), and providing more information about the program throughout the college (1). 

Student Interviews 

Personal interviews were held with nine students on site at Glendale Community College in the Learning 
Center. Topics covered included the method of use, best and least liked features, problems with use, and 
miscellaneous comments. 

All nine students said that they used LEE on their own, but with teacher supervision (in the Learning 
Center). Three of the students said that the exercises and instruction were what they liked best about the program, 
while the slow response time of the computer was what they liked the least. Difficulty using the mouse as well as 
starting and ending the program were cited as problems with the program. General comments were favorable overall. 
Most students interviewed said they liked the program, learned a lot from it, and would use it again. 

Test Scores 

Module-end performance tests were completed by five students. Not every student completed every section, 
so only those sections with scores are reported in Table 3. The tests consist of sections A, B, and C, section A 
having the easiest items and section C having the most difficult. Test results are given based on the percentage of 
items correct out of the total number of items in a section. A mean score is given for each section. 

Observational Data 

Observational data were gathered at the Learning Center at Glendale Community College over a span of four 
hours one morning. The Learning Center has several sites for Learning English Electronically and nine students 
used the program during the observational time period. Evaluator observations are reported in Table 4. 

Discussion 

The dominant finding from this evaluation is that both teachers and students think that Learning English 
Electronically is a good program. Both groups expressed very positive attitudes toward it. 

Module-end performance test scores indicated that most students found that Section A was challenging for 
them, but that they could still obtain a moderate level of success completing it. (The mean score for this section 
was seventy-three percent.) While Sections B and C were not as frequently completed due to the level of difficulty, 
they were successfully accomplished by a few students. (The mean scores for these sections were both seventy-five 
percent, but one student did score a hundred percent on each section.) Regarding these data, one suggestion for 
improvement would be to perhaps make some easier test items for each level so that the students may feel a greater 
level of success in their overall scoring, but yet be just as well prepared to continue on to the next sections of the 
test. 

Other areas of the program that were cited as needing improvement are listed below: 

- the computer response time 

- the sound 

- the printing capabilities 

- the instructions to start and end the program 

- the instructions for the mouse 

In summary, the evaluation of Learning English Electronically revealed that the program is well liked by 
both students and teachers. Several recommendations are offered for potential improvements in the program based on 
student and teacher attitudes and suggestions, as well as observational data and performance test scores. LEE is a 
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well regarded program and these recommendations simply offer the potential to further refine it and contribute to it’s 
development. 

Table 1. Survey Responses From Teachers 

SA = strongly agree 
A = agree 
N = no opinion 
D = disagree 
SD = strongly disagree 





SA 


A N 


D 


SD 


Mean 


1. The modules are well organized. 


3 








2.00 


2. The skills taught in the modules 
1.33 

are valuable for my students. 




2 1 








3. The modules are useful. 


1 


2 






1.67 


4. The stories aid in the learning process. 


2 


1 






1.33 


5. The rules are clearly stated. 




3 






2.00 


6. Instructions for the exercises are clear. 


3 








2.00 


7. There are enough practice exercises. 


2 


1 






1.33 


8. The modules are about the right length. 


3 








2.00 


7 17 

9. What are the major strengths of this program? 

Additional practice if the students take advantage of it. (1) 
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Self-paced, nonthreatening. It has sound. (1) 

Grammar topics with content topics. (1) 

10. What are the major weaknesses of the program? 

The slowness. (1) 

No opinion. (1) 

1 1 . What could be done to improve the program? 

Make more students aware of it. (1) 

Improve its speed. (1) 

No opinion. (1) 

12. How do you use this program in your class? 

a. as a supplement b. as the main part of the class 

3 0 

13. Would you use this program again? 

Yes. (2) 

Always recommends extra services at Learning Center to all students. (1) 

14. Other comments or suggestions? Needs to be completed. (1) 





Table 2. Survey Responses From Students 



SA = strongly agree 
A = agree 
N = no opinion 
D = disagree 
SD = strongly disagree 



1 . The program was easy to use. 




SD 

2 


A 

6 


u 


D 


SD 

1 


Mean 

2.11 


2. The sound with the story was helpful, 
I liked it. 




2 


4 


1 






1.86 


3. The practice exercises helped me learn. 




5 


3 








1.38 


4. The instructions were easy to 
understand. 




2 


5 


1 


1 




2.11 


5. Going back to the instructions helped 
me to learn. 




3 


3 


1 






1.43 


6. The stories helped me to learn. 




4 


4 








1.50 


7. 1 learned a lot from the program. 




4 


4 








1.50 


8. The review helped me. 




3 


3 


2 






1.88 


9. 1 liked the way the program told me 
the answers. 




4 


4 








1.50 


10. 1 liked using the program. 




6 


2 








1.25 


- 




35 


38 


5 


1 


1 


1.65 


11. The module was: 
















a. too long 
1 


b. about right 
5 




c. 


too short 
1 








12. Modules on the average were: 
















a. too hard 


b. about right 




c. 


too easy 









7 



13. 1 used the modules because the teacher assigned them: 



a. yes 
4 



b. no 
2 
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14. 1 used the modules on my own: 

a. yes b.no 

4 3 

15. What did you like best about the program? 

The instruction helped me to learn. (1) 

Easy to understand. (1) 

Using the computer. (1) 

16. What did you like least about the program? 

Too slow. (3) 

No printing when desired. (1) 

17. What could be done to improve the program? Other comments or suggestions? 

More information on it throughout the college. (1) 

Make response time faster. (2) 

More exercises and more tests. (1) 



Table 3. Module-End Performance Test Scores 
Section A 

100 % 

70% 

70% 

50% 

Mean 73% 



Section B 
100 % 



50% 

75% 





Section C 
50% 



100 % 

75% 
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Table 4. Evaluator Observations 



Evaluator observations are as follows: 

- first time users of the program need a lot of guidance from the 
instructors available in the lab 

- repeat users are able to manage with minimal teacher assistance 

- the program does work quite slowly and that seems to frustrate some of 
the students 

- the students don't rely on the written guide booklet very much 

- since the students can not print out the rules or all the exercises, they 
bring notebooks and take notes 

- students use the option of reviewing the rules in the program quite often 

- students who used the sound option in the program couldn’t hear it very 
well and needed to use headphones 

- students who worked on the lessons about modals became frustrated 
because the computer only gave one answer as correct, when in reality 
more than one correct answer exists. Some proofreading and correcting 
also needs to be done as there are mistakes in some of the sentences. 
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Practical Motivational Techniques for Preservice Teachers and 
Instructional Design Strategies 



Heidi L. Schnackenberg 
Arizona State University 



This paper describes instructionally designed educational units for preservice teachers. The units pertain to 
specific practical motivational techniques for preservice teachers to use in their future classrooms. A description of 
the units, as well as field test data (posttest scores and attitude survey results), will be provided during the proposed 
presentation. Implications of the compiled data, and how they relate to the instructional design, will also be 
discussed. 

Purpose and Rationale 

The primary goal of teachers is to educate their students. The learning process is a conscious and deliberate 
act that requires students to expend effort and focus their attention. It is the job of the teacher to provide a context in 
which students choose to learn. The context, therefore, needs to be motivational in nature. Motivation is an 
integral part of learning because it is what drives students to engage fully in educational opportunities. Essentially, 
without motivation in some form, learning does not occur. And if learning does not occur, teachers are not 
accomplishing their goal. 

A Practical Guide to Motivating Your Students: What the Textbooks Never Tell You! is a program 
designed to help future teachers learn to use various practical motivational techniques to motivate students in grades 
K-12. The issue of student motivation is not a new one, nor is it one that has all- encompassing solutions. But it 
is certainly an issue of interest to future teachers and their potential success as educators. This unit offers some 
insight and recommendations on student motivation for the target audience. 

Materials 

A Practical Guide to Motivating Your Students: What the Textbooks Never Tell You! consists of all the 
materials needed for instruction and assessment on the program content and objectives. These include an instructor 
guide, a set of transparencies corresponding to each unit, a forty-three page student guide, and a pretest and posttest 
for each student. The materials are described below: 

• The instructor guide includes an introduction of the program, including objectives, organizational 
format, materials, lesson sequence, planning/presentation methods, and assessment instructions. 
Optional supplementary activities are also included at the end of each unit. Information about practical 
motivational techniques can be obtained from reading the Student Guide. 

• The student guide contains readings pertaining to the techniques being taught. Practical application 
examples are included in these readings. Outlined lecture notes are included to assist the students with 
lecture information. Worksheets and diagrammed group activities are contained in the student guide as 
well. 

• Overhead Transparencies are provided for each unit to assist the instructor in presenting the 

information. 



Objectives 

A Practical Guide to Motivating Your Students: What the Textbooks Never Tell You! is designed so that 
students will attain three learning objectives. Each unit covers each of the learning objectives. The objectives are 
listed below: 

1. The students will name four practical motivational techniques. 

2. The students select the strategy that exemplifies a motivational technique, given descriptions of 
classroom strategies and motivational techniques. 

3. The students describe how to implement specific motivational techniques in given classroom scenarios. 
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4. The students also learn several definitions of motivation. 



Tryout Description 
Participants 

The program was field tested with twenty-four undergraduate education majors from a large southwestern 
university. All of the students were currently enrolled in a course in learning and motivation. The instructor is a 
teaching assistant for the course at the same university. 

Process 

The instructor taught the program to the tryout group using the procedures described in the Instructor Guide 
to A Practical Guide to Motivating Your Students: What the Textbooks Never Tell You!. The pretest and posttest 
was administered and scored by the instructor, as is the standard practice during regular instruction with the program. 
The instructor also completed an Instructor Reaction Sheet at the end of the program and administered the Student 
Reaction Sheet to the tryout group. 

The program was originally designed for use over several days, but the field test occurred in one evening. It 
consisted of three hours of in-class instruction, including pretest, lectures, practice exercises, review and posttest. 

Data sources and collection 

Four different instruments were used to collect data from the program: I) student pretest, 2) posttest, 3) 
student attitude survey, and 4) instructor attitude survey. 

The pretest and posttest are 20-item parallel-form tests that consist of multiple choice and constructed- 
response questions that cover each of the three objectives. The Instructor Reaction Sheet and the Student Reaction 
Sheet were constructed to collect data from the field test that would be useful for evaluating the program and for 
identifying appropriate revisions in it. 

Results 

Test performance 

The mean percentage scores of the field-test students on the pretest and posttest are shown by objective in 
Table I. The table reveals that the students’ overall mean scores were 70.83 percent on the pretest and 96.36 percent 
on the posttest. Students’ pretest scores were 75.00 percent or below on the first two objectives, and 81.25 percent 
on the third objective. Posttest scores were 95.75 percent or above on the first two objectives, and 94.75 percent for 
the third objective. 
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Table 1. Pretest and Posttest Mean Percentage Scores by Objective 



Objective 


% 

Pretest 

Mean 


% 

Posttest 

Mean 


1 . The students will name four practical 
motivational techniques. 


56.25 


95.75 


2. The students select the strategy that 
exemplifies a motivational technique, 
given descriptions of classroom strategies 
and motivational techniques. 


75.00 


98.58 


3. The students describe how to implement 
specific motivational techniques in given 
classroom scenarios. 


81.25 


94.75 


Mean Totals 


70.83 


96.36 



Student attitudes 

Student responses to the items on the Student Reaction Sheet are summarized in Appendix A. It can be 
seen that one-hundred percent (24 of 24) of the students thought that they learned important things about increasing 
student motivation and liked each of the units. Also, ninety-six percent (23 of 24) of the students indicated that the 
program was not too hard. When asked what the most important things that they learned from the program were 
62.50 percent (15 of 24) of the students said "learning the different techniques". When asked what they would change 
about the units, 41.67 percent (10 of 24) of the students responded "nothing". Several students left the open-ended 
questions blank and were recorded as having “no comment”. 

Instructor attitudes 

The responses of the teacher on the Instructor Reaction Sheet are summarized in Appendix B. The teacher 
reported that the students reactions to the program were generally positive, but a few students thought that reviewing 
the answers to the practice exercises was tedious. The teacher was also satisfied with the progress of the students 
from pretest to posttest and said that the program works well in teaching various motivational techniques. Finally, 
the teacher said that the content was appropriate for the target audience and responded “yes” to the question “Would 
you teach this program again?” 

Revisions 

Tryout data revealed that scores on the student pretests were high (70.83 percent). In most cases, this may 
indicate that either the items contained on the test were too easy or that the students had already mastered the content 
in the program. In this situation, the students had been observing practicing teachers and their motivational 
techniques prior to the field test of the program. This exposure to actual implemented motivational techniques may 
explain the higher scores on the pretest data. 

Other tryout data indicates that 16.67 percent (4 of 24) of the students did not like listening to the lectures. 
If future clients wish, the lectures may be omitted from the program, provided the lecture notes remain. Also, 20.83 
percent (5 of 24) of the students thought the program was too repetitive. Sequencing of lectures, practice exercises, 
and activities will be modified to provide more variety. 

The only recommendation for a revision identified by the instructor consisted of including more examples to 
give to the students. This change will be included into the final version of A Practical Guide to Motivating Your 
Students: What the Textbooks Never Tell You!. 
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Conclusions 

A Practical Guide to Motivating Your Students : What the Textbooks Never Tell You! appears to be very 
effective in teaching undergraduate education majors about various practical motivational techniques to use in their 
future classrooms. Test results indicate that the students learned the information necessary to make them successful 
with these concepts. Student Reaction Sheets revealed that the students also enjoyed themselves while they were 
learning. 

The instructor was able to use the materials in an efficient and professional manner in order to allow for 
effective instruction and maximum facilitation of learning. 
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Appendix A 



A PRACTICAL GUIDE TO MOTIVATING YOUR STUDENTS: 
WHAT THE TEXTBOOKS NEVER TELL YOU! 



RESPONSES TO STUDENT REACTION SHEET 
(Responses for 24 students.) 



A. How much did you like each of the activities listed below? 





I liked 


I liked 


I did not 




this a lot 


this OK 


like this 


1. Taking the pretests and posttest. 


3 


20 


1 


2. Listening to the lectures. 

3. Reading and discussing Unit One: So 


7 


13 


4 


What is Motivation Anyhow? 

4. Reading and discussing Unit Two: 
Keep Difficulty Levels of Tasks 


8 


16 


0 


Balanced I 


9 


15 


0 


5. Reading and discussing Unit Three: 


Make Information Relevant! 


8 


15 


i 


6. Reading and discussing Unit Four: 


Provide Rewards! 


8 


15 


1 


7. Reading and discussing Unit Five: 


Use Variety in Instruction! 


7 


15 


2 


8. Doing the practice exercises in the 


Student Guide. 


7 


14 


3 


B. Answer each question by drawing 


a circle around 


1 YES or NO. 
YES 


NO 


1. Do you think you learned important things 




24 


0 



about practical techniques for increasing 
student motivation ? 

2. Do you think that this program was too hard? 

3. Did you like Unit One: So What is Motivation 
Anyhow? 

4. Did you like Unit Two: Keep Difficulty Levels 
of Tasks Balanced! 

5. Did you like Unit Three: Make Information 

Relevant ! 



1 

YES 



23 

NO 



24 



0 



24 



24 





6. Did you like Unit Four: Provide Rewards ! 



24 



0 



7. Did you like Unit Five: Use Variety in 

Instruction ! 24 0 

C. What did you like best about Unit One: So What is Motivation Anyhow?, Unit Two: Keep 
Difficulty Levels of Tasks Balanced !, and Unit Three: Make Information Relevant!. 

Learning the different techniques. (13) 

No comment. (4) 

Examples. (3) 

How the instructor asked different people their definition of motivation. (2) 

Clearly stated objectives. (1) 

The reviews. (1) 

Thinking of teachers that were motivational. (1) 

Tallying the different reasons why the class thought this program was relevant to them. (1) 
Exercises. (1) 

D. What did you like best about Unit Four: Provide Rewards! and Unit Five: Use Variety in 
Instruction /. 

Learning the different techniques. (8) 

Examples. (5) 

No comment. (5) 

Discussing different rewards for different age groups. (2) 

Switching desks. (2) 

The situational (scenario) questions. (1) 

Exercises. (1) 

E. What do you think are the most important things you learned from Unit One: So What is 
Motivation Anyhow?, Unit Two: Keep Difficulty Levels of Tasks Balanced!, and Unit 
Three: Make Information Relevant /? 

Learning the different techniques. (15) 

No comment. (7) 

Thinking of teachers that were motivational. (2) 

F. What do you think are the most important things you learned from Unit Four: Provide 
Rewards! and Unit Five: Use Variety in Instruction!? 

Learning the different techniques. (15) 

No comment. (7) 

Discussing different rewards for different age groups. (2) 

G. What do you think should be changed or added to make Unit One: So What is Motivation 
Anyhow?, Unit Two: Keep Difficulty Levels of Tasks Balanced!, and Unit Three: Make 
Information Relevant! better? 

Nothing. (10) 

Make (the program) less repetitive. (5) 

Be more in depth with the techniques. (2) 

Do not over-do the questions. (1) 

No comment. (6) 3 O g 
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M. What do you think should be changed or added to make Unit Four: Provide Rewards! and 
Unit Five: Use Variety in Instruction! better? 

Nothing. (10) 

Make (the program) less repetitive. (5) 

Be more in depth with the techniques. (2) 

Do not over-do the questions. (1) 

No comment. (6) 





Appendix B 



A PRACTICAL GUIDE TO MOTIVATING YOUR STUDENTS: 
WHAT THE TEXTBOOKS NEVER TELL YOU! 



RESPONSES TO INSTRUCTOR REACTION SHEET 

1 . How clear were the lesson procedures in the Instructor Guide? 

Clear 

2. How easy or difficult did you find A Practical Guide to Motivating Your Students: What 
the Textbooks Never Tell You! to teach? 

Easy 

3. How satisfied were you with the progress of most of the students from pretest to 
posttest? 

Satisfied 

4. How appropriate was the content in A Practical Guide to Motivating Your Students: 
What the Textbooks Never Tell You! for the target audience? 

Very appropriate 

5 . How well do you think A Practical Guide to Motivating Your Students: What the 
Textbooks Never Tell You! helps students to understand the various practical 
motivational techniques covered in the Student Guide? 

Excellent 

6 . What was the reaction of the learners to A Practical Guide to Motivating Your Students: 
What the Textbooks Never Tell You /? 

The students enjoyed the program. Some of the students seemed to think that reviewing the answers to every 
practice exercise was a bit tedious, but that may be because they’re not used to comprehending material so 
easily. 

7 . How many students participated in the field test for A Practical Guide to Motivating 
Your Students: What the Textbooks Never Tell You!? 

24 students. 

8. How many sessions did you spend teaching A Practical Guide to Motivating Your 
Students: What the Textbooks Never Tell You!? 

1 session. 

About how long was each session? 

3 hours long. 
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9. What content if any would you delete from the A Practical Guide to Motivating Your 
Students: What the Textbooks Never Tell You!? 

None. 

10. What, if anything would you add to A Practical Guide to Motivating Your Students: 
What the Textbooks Never Tell You!? 

Include more examples so that the instructor has a choice of which ones to tell to the students. 

11. Would you teach this program again? 



A Qualitative Look at Preservice Teacher’s Perceptions of the Future 

of Computers in Education 

Heidi L. Schnackenberg 
Wilhelmina C. Savenye 
Arizona State University 



The use of computers in education is a relatively new realm. There is no scientific basis for predicting its 
future. Although numerous studies focus upon people’s attitudes toward the present-day use of computers in 
education, relatively few studies analyze people’s perceptions of how computers will be used in education in the 
future. Perhaps the best source of information about the future of computers in the field of education is the opinions 
and perceptions of future educators. 

Research in the area of attitudes toward computers has been extensive in the last several years. Ellsworth 
and Bowman (1984) found that students who were exposed more to computers were more likely to develop positive 
attitudes about them. Arndt, Clevenger, and Meiskey (1985) reported that students who viewed computers in a 
positive manner possessed more computer experience than those who viewed computers in a negative manner. 
Temple and Lips (1989) reported that gender also influenced students attitudes toward computers. They found that 
undergraduate males reported feeling more comfortable using computers than undergraduate females did. Smith 
(1987) found that elementary students were more confident than high school students in their ability to use 
computer. And Campbell reported that student self-assessment of computer literacy skills affects enrollment in 
computer courses (Campbell-abstract). 

Recently, several research studies have explored the perceptions of professionals and students on aspects of 
the field of educational technology. Sullivan, Igoe, Klein, Jones, & Savenye (1993) studied the opinions of 
university faculty, graduate students, and personnel employed in training positions on the future of the field of 
educational technology. A survey composed of six topic areas was used to gather the data. Two of the six topic 
areas investigated were: Technology and Individualized Instruction and Advances in Technology. 

In respect to Technology and Individualized Instruction , respondents showed relatively high overall 
agreement with statements such as “Computer-based instruction will result in much greater individualization of 
instruction” and “Computer-delivered instruction will benefit individual students by enabling them to manage their 
own learning to a greater extent”. Master’s students had significantly greater agreement than faculty that CBI will 
result in much more individualized instruction. 

Regarding Advances in Technology , respondents did not show strong general agreement with statements 
such as “Technology-as-hardware will have a greater influence in education and training than technology as 
instructional systems design”, “The role of instructional designers will shift from designing instruction to creating 
systems that design it”, and “Expert systems will design effective instruction sequences or programs with minimal 
human input.” A significant difference was found only on this last item in that master’s students showed stronger 
agreement than either doctoral students or faculty members. 

The purpose of the present study was to determine the perceptions of preservice teachers on how computers 
will be used in schools in the future. Subjects were university students. These students represent a less experienced 
and educated group than Sullivan, Igoe, Klein, Jones, & Savenye’ s (1993) faculty, graduate students, or trainers. 

Method 

Subjects 

Participants in the study were 40 undergraduate students enrolled in an introductory computer course at a 
large southwestern university. The majority were education majors with varying concentration areas, but students 
from other academic disciplines participated as well. Computer experience varied among the subject population. 

Procedures 

Students were given a sixty minute presentation/demonstration on how computer and multimedia 
technologies are used in schools. After a brief introductory overview of what multimedia is and how it is used in 
education, students were shown a 10- minute video about the TLTG interactive video physical science curriculum. 
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Students were then given demonstrations of several HyperCard-based CAI programs. Hands-on experience was not 
included in the presentation, only overhead visuals, video, and verbal explanations. 

After the presentation, students were told to discuss in groups for 10 minutes ways in which computers will 
be used in schools 10 years from now. A group spokesperson then gave a 15 minute report on the ideas and 
opinions formulated in his/her group’s discussion. 

A follow-up project was then assigned to the class. Individually, the students were to write a one-page 
essay addressing the question previously discussed in class. The papers were collected one week later in the next 
regular class session. Extra course credit was given to each student that turned in a paper. 

The study was qualitative in nature. Student responses were categorized and the categories were collapsed. 

Results 

Data gathered from student essays is shown in Table 1. 37 students stated that they thought that 

assignments and materials would be accessed via e-mail or the internet. 32 students indicated that they thought that 
software would be personalized, self-paced, and written by students, while 31 said that virtual reality, CD roms, and 
holograms would be commonplace. 30 students stated that they thought that computers would be voice-activated and 
have touch-screens, while 25 students stated that they thought the teacher would still be the center of the classroom 
and computers would be used for grades and attendance records. 23 students volunteered that they thought that every 
student would have a computer and 18 students stated that computers will replace books and floppy disks will 
replace paper. 13 students wrote that they thought computers will control fire drills, demonstrations, and career 
exploration. 1 student each felt that computer equipment will not be expensive and computer access will be limited. 
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Table 1. Perceptions of Computers Usage in Schools 10 Years in the Future 



Perception Number of Responses 

Assignments, materials, library information, 37 

gotten from the internet and e-mail 

Students write their own personalized, 32 

self-paced, software for all subjects. 

Virtual reality, holograms, CD Roms, 31 

real-time capabilities are commonplace. 

Computers are voice-activated, have touch-screens, 30 

recognize thumbprint ID's , and speak. 

Teachers are the center of the classroom and use a 25 

(hand-held) computer for grades and attendance. 

There is 1 computer for every student, even in poor 23 

and rural school districts. 



There are portable computers instead of books, 1 8 

floppy disks instead of paper, videos instead of field 
trips, computerized chalkboards, and lightpens. 

Computers are used for fire drills, demonstrations, 13 

speakers, tutorials, research, creativity development, 
career exploration, and word processing. 

Computer equipment is not very expensive. 1 

Access to computers and software is limited. 1 



N = 40. Students could mention a number of responses. Categories were condensed. 

Discussion 

The present study examined the perceptions of preservice teachers on how computers will be used in schools 
10 years from now. Many students stated that they thought assignments and materials would be accessed via e-mail 
or the internet, software would be personalized and self-paced, computers would be voice-activated and have touch- 
screens, and virtual reality, CD roms, and holograms would be commonplace. Other students stated that they thought 
that every student would have a computer, computers will replace books and floppy disks will replace paper, and the 
teacher would still be the center of the classroom. And a few students thought computers will control fire drills, 
demonstrations, and career exploration, computer equipment will be inexpensive, and computer access will be 
limited. 

The overall findings of this study support the findings of Marcinkiewicz (1994). He states that pre-service 
teachers have high expectations of using computers in their classrooms. Statements made by the students in this 
study, such as “there will be one computer for every student” and “assignments will be accessed via e-mail”, also 
reveal high expectations for the use of computers in the classroom. However, the original question that students in 
the present study had to answer involved the future of computers in the classroom 10 years from now. Perhaps some 
of their expectations were too high for that time frame. 
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Students receiving assignments from an on-line network, floppy disks replacing paper, and every student 
having their own computer may certainly happen in education in the future, but the likelihood of them happening in 
the next 10 years is doubtful. Granted, some students in some computer classes are required to retrieve their 
assignments on line, but for this to become commonplace it will necessitate all students having an individual 
terminal. According to O’Donnell (1993) “the cost of computer systems has dropped to the point where almost 
anyone who desires to own a computer can now afford one”. Unfortunately, public education isn’t for almost 
anyone, it’s for everyone. Until computers are inexpensive enough for school systems to purchase one for every 
student, assignments will still be given by the teacher and paper will be more widely used than floppy disks. 

While some of the students perceptions may have been a bit beyond the requested time-frame, other 
perceptions were more accurate. Many of the resources the students thought would be in place in schools in 10 years 
are happening now. Many schools presently have information accessible in CD rom format, voice and touch-screen 
activated computers for physically challenged students, and track grades and attendance via computer. While these 
capabilities are not as universal as they may become, they certainly are in place at the present time. Whether the 
students in this study said that these capabilities were futuristic was due to a keen awareness of what is going on in 
schools today or the complete opposite is unknown. It is perhaps a bit of both. 

Two of the more popular responses that seem to conflict are that teachers will remain the center of the 
classroom (25 responses) and students will write their software to teach themselves at a more personal, self-paced 
level (32 responses). By allowing students to write their own instructional software would permit them to make 
their own instructional choices. Allowing students to make their own instructional choices has been termed “learner- 
control”. 

The effects of learner-control on student achievement is inconclusive. Some research results indicate that 
individuals learn more when given control over their instruction. Ross, Morrison, & O'Dell (1989) reported that 
higher posttest scores were obtained by undergraduate education majors who were allowed to select the instructional 
presentation medium than by students who were not. Other research indicates that individuals learn less effectively 
when given control over their instruction. Pollack and Sullivan (1990) found that seventh-grade science students 
receiving required practice items had higher posttest scores than students allowed control over the amount of practice 
they received. Until the effects of learner-control on achievement can be conclusively determined, it may be better to 
allow teachers to write instructional software instead of students, regardless of the technological capabilities of the 
time. The implication of a recommendation such as this one is that until research can prove that they are not 
beneficial to education, teachers should remain at the center of the classroom. 

Further research on educators perceptions of the future of computers in their field could include comparisons 
between preservice teachers ideas of how computers are used in the classroom today and how practicing teachers 
actually use them. Also, long-range data could be gathered on how preservice teachers estimate they will use 
computers in their future classrooms and then on how they do use them when they become teachers. Research such 
as this can help us to better determine the direction computers will take in education in the future. 
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Purpose and Rationale 

Much of what people learn from outdoor educational settings, i.e. zoos and gardens, is based on what they 
see and read. Almost all of the communication in places such as zoos and gardens stems from the exhibits 
themselves or signs about the exhibits. Often, the expressiveness and educational power of an exhibit depends upon 
the effectiveness of the form of visual communication being utilized. Therefore, evaluation of the various forms of 
visual communication in outdoor educational settings is necessary in order to determine the effectiveness of 
exhibitions, educational activities, and/or conservation programs. 

The mission of The Phoenix Zoo is to “inspire people to live in ways that promote the well-being of the 
natural world”. As part of it’s Strategic Plan, The Zoo intends to integrate the collection of visitor data for decision- 
making throughout all phases of development. In order to do this, The Zoo secured the services of the authors of 
this paper to conduct a small-scale preliminary study to determine visitor use and satisfaction related to one 
component of The Arizona Trail, The Reptile Section. Study data were collected to answer the following questions: 

1 . How often do visitors to the Reptile Section of The Arizona Trail visit The Zoo and where do they come 

from? 

2. What are their reasons for visiting The Zoo and this Trail? 

3. How clearly do visitors understand that they are entering The Arizona Trail and the Reptile Section? 

4. What are visitors’ expectations of their visit to the Reptile Section? 

5. What is their interest in reptiles? What do they most want to know about reptiles? 

6. What do visitors learn from their experience with the Reptile Section? What do they most want to 
know? Is their anything they wished they had learned, but didn’t? 

7. What do visitors think about the Reptile Section and what suggestions do they have for improvements? 

8. How long, on average, do visitors spend viewing the inside and outside components of the Reptile 
Section exhibits? 

9. How do visitors use the Reptile Section? What behaviors do they engage in during their visit? 



Method 

The study was conducted during the month of July, 1995, during all hours of the day, weekdays and 
weekends, with most data collected during the mornings. Data were collected using questionnaires, interviews, and 
observations of visitors. These methods are described below. It should be noted that the weather was typical for 
Phoenix during this season, that is, generally sunny and hot. 

Pre-Viewing Surveys 

A sample of visitors was interviewed immediately after they entered The Arizona Trail to determine their 
preconceptions and expectations about viewing the Reptile Section. An eight-item questionnaire called the “Pre- 
viewing Survey” was developed for this purpose. Visitors answered questions about the frequency of their visits to 
the Zoo, where they live, understanding of The Arizona Trail, reason for visiting The Zoo and The Arizona Trail, 
clarity about the Reptile Section, expectations of their visit to the Reptile Section, and their interests related to 
reptiles. 

Post-Viewing Surveys 

In support of the Zoo’s mission to communicate effectively with visitors about the natural world, a sample 
of visitors was interviewed after they had viewed the Reptile Section to determine their attitudes toward the exhibit, 
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what they felt they learned from it, and visitor suggestions for improving the Reptile Section. A nine-item 
questionnaire was developed for this part of the front-end evaluation study. Visitors were asked what they found out 
from viewing the Reptile Section, what they most wanted to know about reptiles, and what they wished they had 
learned about reptiles from this exhibit. They were also asked to rate the exhibit and describe whether it caught their 
attention, whether they found the reptiles easy to see, and their perceptions about the signs in the exhibit. Finally, 
visitors were asked about their suggestions for helping The Zoo improve the Reptile Section. 

Targeted Visitor Observations 

Two types of targeted observations were conducted: 

Time Spent Viewing the Reptile Section - A large sample of visitors was observed to determine the 
average time visitors spent, during summer, viewing the inside and outside sections of the exhibit. 

How Visitors Use the Reptile Section - Another large sample of visitors was observed carefully to 
determine the types of behaviors visitors engage in while they visit the Reptile Section. 

Findings 

. In this paper, a summary of the findings is reported. The presentation will summarize all of the findings. 

What did visitors learn from the Reptile Section? 

In response to the question, “What are the main things you found out from viewing the Reptile Exhibit?” 
on the post-viewing survey, visitors described various aspects of the exhibit. It is worth noting that about one- 
fourth (8 of 30) of these visitors described that they learned the difference between “venomous” and “poisonous”, 
information highlighted by a sign made by a local elementary school class. Many visitors described things they had 
learned about snakes by reading the exhibit signs, including, how many different kinds of rattlesnakes there are (4), 
what snakes eat (3), where snakes live in Arizona (2), how big the snakes are (2), which snakes are venomous and 
which are not (1), and about snakes’ habitat (1). A few visitors mention they’d learned the names of the animals (3), 
what different animals look like (1), and about scorpions (1). Finally, four visitors said they didn’t find out anything 
in particular, but had just looked at the animals (4), and one said he or she had learned nothing new, having taken a 
tour of the exhibit earlier. 

What do visitors think about the Reptile Section and what suggestions do they have for 
improvements? 

When asked what they thought of the signs in the Reptile Section, sixteen of 26 visitors made positive 
comments, including “good” or “okay” (8), informative (5), liked the maps on the signs (2), and easy for children to 
understand (1). Ten visitors made suggestions including that the signs are too small, print is too small, or are hard 
to notice (4), need to be lower in order for them to be easier to read (3); some are missing (1); are boring (1); or 
should say something about diet or living conditions (1). 

Visitors were also asked their suggestions for helping The Zoo improve the Reptile Section. Nineteen 
suggestions were made, while 13 visitors responded, “None”. Suggestions were relatively specific (and sometimes 
conflicting) and some of them included: 

Suggestions about signs: 

- make signs bigger and more attractive (similar to venomous/poisonous sign) (3) 

- put signs down lower so that they’re easier to see (1) 

- get talking systems for people who have problems reading 

- put lights on signs in order to see them better 

Suggestions about snakes: 

- tell the age of snakes (1) 

- get more snakes (1) 

- have less snakes and put them all in one tank (1) 

- and bring out snakes more often to see and touch (1) 
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How long, on average, do visitors spend viewing the inside and outside components of the 
Reptile Section exhibits? 

The 101 visitors who viewed in small groups or as individuals toured the Reptile Section in groups of from 
one to eight, with most groups consisting of two or three visitors. They viewed the inside of the Reptile Section 
for an average of 2.94 minutes, with a range of 10 seconds to 10 minutes and 25 seconds. They viewed the outside 
of the exhibit for an average of 2.26 minutes, with a range of 15 seconds to 8 minutes and 5 seconds. (It may be 
useful to note that 30 seconds is considered a fairly good average viewing time for museum exhibits, although the 
two halves of the museum are larger than many museum exhibits.) 

How do visitors use the Reptile Section? What behaviors do they engage in during their 
visit? 

Visitors viewed the exhibits in many different ways. For instance, on the inside, without the railing, they 
got up close to the glass to see animals (13). Both inside and outside, visitors picked children up to see upper cases 
(4); just glanced at cases (3); leaned down to see cases (2); read signs silently (2); never looked at the exhibit while 
they walked past (2); tapped on the glass (1); or climbed on the railing to see cases (1). 

Summary and Conclusions 

Based on the collected data, most visitors did not indicate a specific interest in learning particular 
information about reptiles, although most indicated they did have some interest in reptiles, particularly snakes. A 
few did mention specific things they wished to learn about reptiles from their visit, as well as suggestions for 
improving the forms of visual communication in the exhibit. 

Of the thirty visitors interviewed after they viewed the Reptile Section, many indicated they had learned 
many things, mostly about snakes, from their visit, describing specific facts, such as that there are different 
rattlesnakes and where they live in Arizona. It is interesting to note that by far the fact most frequently described by 
visitors was the information contained on the school-group-developed sign about the difference between “venomous” 
and “poisonous”. This particular sign, in contrast to the others contained in the exhibit, contained a clear, simple 
message with a thought-provoking question at the top. It would appear that the design of this particular form of 
visual communication was more effective than the design of the smaller signs contained in the exhibit. 

During observations it appeared that visitors do not read the small signs accompanying the exhibits, 
possibly because many of the signs are quite high and the print is small, as visitors mentioned later. Visitors were 
often observed, however, reading the names of the reptiles aloud. While most visitors made positive comments 
about the forms of visual communication in the reptile exhibit, almost half suggested improvements, such as 
making the signs bigger or lower, making the print bigger, or including more interesting information. 

The results of this study suggest several design implications for visual communication in outdoor 
educational settings. Several guidelines are listed below to help designers improve the effectiveness of visual 
communication in these types of settings in the future: 

1) Arouse learner curiosity with a simple, yet thought-provoking question or title. 

2) Below the opening question, have a clear, succinct answer. 

3) Use simple, clear language. 

4) Use large, clear text. 

5) Do not put too much text on any one sign. 

6) Utilize pictures and graphics to help learners see to what the text is referring. 
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Instructional Programs 



Heidi L. Schnackenberg 
Howard J. Sullivan 
Arizona State University 



Introduction 

The idea of learner control over instruction has enjoyed increasing popularity as a result of the growth of 
computer-assisted instruction in the schools. Several researchers have investigated the effects of allowing learners to 
choose the amount of practice, feedback, and review they desire as they progress through computer-assisted 
instructional programs (Carrier, Davidson, Williams, & Kalweit, 1986; Hicken, Sullivan, & Klein, 1992; Kinzie, 
Sullivan, & Berdel, 1988). Other researchers have allowed learners to choose the method of instructional delivery, 
such as lecture or group discussion (Pascal, 1971), or the size of the group in which they wish to work (Peterson & 
Janicki, 1979). Still others have explored learner control by matching student preferences for amount of instruction 
with the amount they receive (Freitag & Sullivan, 1994; Hannafin & Sullivan, in press). 

Popular argument states that learner control is intrinsically appealing because it allows learners to tailor 
elements of instruction to their individual needs and preferences. Steinberg (1977) claimed that learner control can 
alleviate boredom, anxiety, and frustration, while maintaining learner attention and increasing motivation . 
Nevertheless, research has yielded inconsistent results regarding the benefits of learner control on learner 
achievement. 

Some research results indicate that individuals learn more when given control over their instruction. Ross, 
Morrison, & O'Dell (1989) reported that higher posttest scores were obtained by undergraduate education majors who 
were allowed to select the instructional presentation medium than by students who were not. Kinzie, Sullivan, & 
Berdel (1988) found that eighth-grade science students given control over reviewing content scored higher on a 
posttest than students who were not given this option. Gray (1987) reported that college students having control 
over the sequencing of instructional content in an introductory sociology class scored higher on a retention measure 
than students without sequencing control. In Tennyson's research, twelfth-grade students enrolled in a psychology 
class benefited from controlling elements of their instruction, but only when informed about their own particular 
strategies for learning a task (Tennyson & Buttrey, 1980). 

Other research indicates that individuals learn less effectively when given control over their instruction. 
Carrier etal. (1984) found that seventh-grade learners make poor instructional choices when encountering complex 
instructional material or lacking prior knowledge. Ross and Rakow (1981) reported that college students in an 
introductory sociology class who were given instructional control, but no guidance, also made poor instructional 
choices. Pollock and Sullivan (1990) found that seventh-grade science students receiving required practice items had 
higher posttest scores than students allowed control over the amount of practice they received. 

Hicken et al. (1992), postulated that one reason for the mixed achievement results in studies of learner 
versus program control relates to the differing nature of the instructional programs and learner-control options in the 
studies. In some studies, learners have had the option to add instruction to a relatively “lean” instructional program 
that contains only a basic amount of instruction, thereby lengthening the program and providing themselves with 
more instruction. In such cases, learner control might be expected to be more effective than simply working through 
the basic lean program under program control. In other studies, learner control has involved the option for learners 
to bypass instruction in a relatively “full” instructional program that contains a more comprehensive amount of 
instruction and practice, thereby shortening the program and providing themselves with less instruction. In these 
cases, learner control may not increase the effectiveness of the full program and could conceivably decrease it. That 
is, learner control may have differential effects depending on whether exercising the control enables learners to 
lengthen a basic instructional program or to shorten a more complete one. 

A series of studies (Hannafin & Sullivan, 1995; Hannafin and Sullivan, in press; Hicken et al., 1992; Igoe, 
1993) has been conducted to investigate the effects of learner control in full and lean instructional programs. 
Collectively, these studies indicate that learner control over the amount of instruction in full and lean programs 
mitigates the achievement advantage that would normally be expected to favor the full program over the lean one. A 
significant achievement advantage for the full program over the lean one was obtained in only one of these four 
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studies (Hannafm & Sullivan, 1995). Subjects under learner control in the full programs typically chose to bypass a 
relatively small amount of the instruction in their program version, while those in the lean programs normally chose 
to add less than half of the optional instruction available to them, thereby reducing the difference in the amount of 
instruction that would have been received by the full and lean groups under program control. 

The option-selection pattern noted in the studies above involving learner control in both full and lean 
versions of an instructional program reflects a general tendency for learners to follow the default version of the 
program. That is, learners in a full version of a program tend to bypass a relatively low percentage of optional 
screens, and those in a lean version tend to add a relatively low percentage of optional screens. A similar pattern has 
also been observed by other researchers in studies including only a single learner-control option, to add instruction or 
to bypass it. Generally, learners who have an option to bypass instruction bypass only about 20 percent of the 
optional elements available to them (Pollock & Sullivan, 1990; Lopez & Harper, 1989; Hicken et al., 1992; 
Hannafin & Sullivan, 1995). Learners who have the option to add instruction typically select 30 to 40 percent of 
the additional available elements (Carrier & Williams, 1988; Carrier, Davidson, & Williams, 1985; Kinzie et al., 
1988; Hannafin & Sullivan, in press). An exception to this latter pattern may occur in cases where learners are 
strongly motivated to do well, such as when the learner-controlled program consists of important course content that 
may have a significant influence on the subject’s course grade. Igoe (1993) found that subjects in the lean version of 
an instructional program under such a condition chose to add 70 percent of the additional options available to them. 

Learner time- in-program also varies across full and lean programs, sometimes in a way that is not 
consistent with the variation in the total number of screens viewed. Tennyson (1980) reported that subjects who 
received advisement in an instructional program spent more time and chose more options than subjects given control 
over the amount and sequence of elements in the program. Yet Kinzie and Sullivan (1989) and Kinzie, Sullivan, and 
Berdel (1988) found that learner-control subjects, despite bypassing review sections of an instructional program, 
spent a similar amount of overall time- in-program to subjects who were required to see the review. Interestingly, 
Schnackenberg, Sullivan, Leader, and Jones (1996) found that college students who used a full version of an 
instructional program, containing 242 screens with no learner-control option, did not spend significantly more time- 
in-program than subjects who used a lean version of the same program containing only 158 screens, presumably 
because the subjects in the lean program compensated for their fewer screens by spending more time per screen in the 
program. Hicken et al. (1992), also found that students in the lean version of their program spent significantly more 
time per screen in the program than those in the full version, suggesting that these students were compensating for 
the lesser amount of basic instruction. 

Another factor that may influence the effectiveness of various versions of computer-assisted instructional 
programs is student ability. Ross and Rakow (1981), Tennyson and Rothen (1977), and Goetzfried and Hannafin 
(1985) found that lower-ability students benefit more from program control than from learner control. Hativa (1988) 
reported that lower-ability subjects spent less time-on-task in a learner-control treatment than higher-ability subjects. 
Hannafin and Sullivan (1995) found that higher-ability students in a learner-controlled lean treatment chose to add 
optional elements of the program in 43% of the cases, whereas lower- ability learners chose to add optional elements 
in only 19% of the cases. Thus, lower-ability students may choose to avail themselves of fewer options than 
higher-ability students in learner-control programs, and therefore they may not perform as well under learner control 
as under program control. 

Although learner control studies have yielded mixed results on learner achievement, results related to student 
attitudes and motivation have been consistently favorable. Kinzie and Sullivan (1989) found that high school 
students in a learner-controlled treatment chose to return to that type of program more often than students in a 
program-controlled treatment. Morrison, Ross, and Baldwin (1992) reported that sixth-grade students allowed to 
choose the amount and context of practice problems had more positive attitudes than those who were not. Igoe 
(1993) found that college students given leanPlus and fullMinus versions of a learner-controlled program reported 
positive attitudes toward the learner-control feature in both versions. 

The purpose of the present study is to investigate the effects of learner control and program control in full 
and lean computer- assisted instructional programs on the achievement of higher-ability and lower-ability university 
students. Four versions (program-control lean and full and learner-control lean and full) of a computer-assisted 
instructional program will be used as the instructional materials for the study. The program is designed in the 
Macintosh HyperCard format. Previous research (Igoe, 1993) indicates that subjects in the course involved in this 
study are strongly motivated to perform well in the instructional program and to receive a good grade. 
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The following research questions will be investigated: 

1. Do university students achieve better under program control than under learner control in a computer-assisted 

instructional program? 

2. Do students perform better in a full instructional program than in a lean one? 

3. Does the availability of learner control affect student performance differentially in full and lean programs? 

4. Do higher-ability and lower-ability students perform differently from one another under full and lean programs? 

5. How does option use differ between higher-ability and lower-ability students under learner control in full and 

lean programs? 

Questions related to learner attitudes and time-on-task under the different experimental treatments will also 
be investigated. Time on task will be examined with respect to total time by treatment, mean time per screen, and 
time per screen early and later the program. 

Method 

Subjects 

Participants in the study will be approximately 200 undergraduate education majors enrolled in their first 
semester of a professional teacher preparation program. All participants will be registered for EDP 301, Learning 
and Motivation , during the Fall 1996 semester at Arizona State University. 

Materials 

The instructional materials are designed to teach content from a required textbook, Teaching for Competence 
(Sullivan and Higgins, 1983), for a course in which all students will be enrolled. Three chapters from the text 
(Worthwhile Objectives, Effective Instruction, Assessment) are adapted into a computer-assisted program in the 
Macintosh HyperCard format for the study. The three chapters contain a total of 13 learning objectives. Instructions 
for using the program are included as part of the introduction to the program. 

The program is developed in four versions that represent the four different treatment conditions. The 
elements of instruction other than practice (information, examples, reviews, and summaries) are identical in all 
versions of the program. Each of the 13 objectives is taught through a number of screens which present the 
instruction, practice and feedback, summaries, and reviews. Nine objectives require selected responses in a multiple- 
choice format and four require constructed responses. Practice items consist of multiple-choice questions with two- 
to-four response choices for the nine selected-response objectives and of constructed-response items for the four 
constructed-response objectives. The program tracks each subject's progress by recording each response choice on a 
screen-by-screen basis. 

Program-control subjects advance through the program by using a mouse and selecting a button titled 
"Continue" to go to the next screen. Subjects in the two program-control versions flean and full) are required to 
respond to all screens in their version. The program-control full version contains 174 information screens and 66 
practice-with-feedback screens (six multiple-choice practice items for each of the nine selected-response objectives and 
three constructed-response practice items for each of the four constructed-response objectives). The program-control 
lean version contains the same 174 information screens but only 22 practice-with-feedback screens (two multiple- 
choice practice items for the selected-response objectives and one constructed-response practice item for the 
constructed-response objectives). Thus, there are a total of 240 screens (174 information screens and 66 practice- 
with-feedback screens) in the full program-control version of the program and 196 screens (174 information screens 
and 22 practice-with-feedback screens) in the lean program-control version. 

In the learner-control treatments, the first two practice-with-feedback items for “selected-response objectives” 
and the first one practice-with-feedback item for “constructed-response objectives” are part of the basic program. That 
is, all learners under both full and lean learner-control conditions complete these items. The remaining four practice- 
with-feedback items for each selected-response objective and two practice-with-feedback items for each constructed- 
response objective are optional items under control of the learner. 

In the full version of the learner-control program, a learner begins the practice for an objective by 
completing the two basic practice-with-feedback items (one basic item for constructed-response objectives) for that 
objective. The learner then pushes the “Continue” button to forward the program to more practice-with-feedback 
items for the objective (two more items if it is selected response, one more if constructed) or the “No More Practice” 
button to bypass additional practice on the objective. If the learner chooses to continue at this first choice point, 
she/he is given the same option again (“Continue” or “No More Practice”) after completing the item(s) from the 




initial choice to continue. Thus, learners have a maximum of two choice points per objective, and at their option 
they may complete two, four, or six practice-with-feedback items for each selected-response objective or one, two, or 
three items per constructed-response objective. 

The lean version of the learner-control program differs from the full one in the manner in which students 
choose to add practice or to move on to the next objective. As in the full program, a student begins practice-with- 
feedback for an objective by completing the two basic practice-with-feedback items (one for constructed-response 
objectives) for that objective. The student is then given the option of pushing the “Continue” button, which is 
essentially the default option in the lean program, or the “More Practice” button. The “Continue” button moves the 
student to the next objective without further practice-with-feedback on the current one, whereas the “More Practice” 
button provides two more practice-with-feedback items (one for constructed-response items) on the current objective. 
If a student selects the “More Practice” option at the first choice point, she/he is given the same option (“Continue” 
or “More Practice”) again after completing the item(s) from the first choice. Thus, similar to subjects in the full 
version, learners in the lean one have a maximum of two choice points per objective, and at their option they may 
complete two, four, or six practice-with-feedback items for each selected-response objective or one, two, or three 
items per constructed-response objective. The difference is that in the full program the “Continue” choice moves the 
learner to additional practice-with-feedback items, whereas in the lean program the “Continue” choice moves the 
learner to the next objective. 

Procedures 

Prior to beginning the program, Scholastic Aptitude Test (SAT) or American College Testing Assessment 
(ACT) scores will be obtained for all subjects. A median score will be calculated for the overall sample. Subjects 
with scores at or below the median will be classified as lower-ability, and those with scores above the median will be 
designated as higher-ability. Subjects will then be randomly assigned to one of the four program versions within 
higher-ability and lower-ability groups. 

Each subject will be given an individual program disk with his or her assigned version of the program. 
Instruction sheets with directions for using the available computer facilities on the university campus will be 
included with the disks. Subjects will be given a two-week period without class meetings to complete the program. 
At the end of this period they will report back to their regular class session. The experimenter, several teaching 
assistants, and the course instructor will be available to answer questions about the program. 

A 52-item paper-and-pencil posttest, described in the criterion measures section, will be administered in the 
first class session after the two-week instructional period. Program disks will also be collected at this time. 

Criterion Measures 

The paper-and-pencil posttest will consist of 36 multiple-choice items and eight two-point constructed- 
response items covering the 13 objectives in the instructional program. The multiple-choice items will be scored 
either one or zero and the constructed-response items will be scored either two, one, or zero, according to a scoring 
key developed by the experimenter. Thus, the maximum possible score on the criterion test will be 52. The test 
items will be different items from the practice items, but will be in the same item form as the practice items for each 
objective. 

A thirteen-item attitude questionnaire will assess subjects’ satisfaction with the material, their perceived 
effort, their desire for more information, their continuing motivation and their confidence in their posttest 
performance. The attitude questionnaire, a five-choice Likert-type questionnaire, will be administered on-line 
immediately after students complete the instructional program. 

Design and Data Analysis 

The experimental design is a 2 (learner control mode) x 2 ( full or lean program) x 2 (higher-ability vs. 
lower-ability) posttest-only design with random assignment of subjects to treatments within higher-ability and 
lower-ability groups. Analysis of variance (ANOVA) will be used to analyze the data for achievement and for time- 
in-program. Attitude questionnaire data will be analyzed using a multivariate analysis of variance (MANOVA), 

■ followed by a univariate analysis for each questionnaire item if the multivariate analysis indicates that it is 
appropriate. Option-use data will be analyzed using ANOVA. 
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The Relationship of Media and ISO Theory fThe Unrealized Promise 

of Dale's Cone of Experience 

Barbara Seels 
University of Pittsburgh 



Instructional design theory flows from three major sources: general systems theory, instructional theory, 
and communications theory. The integration of these theories produces instructional systems design (ISD) theory 
(Seels, 1989). Currently, instructional design incorporates theories related to areas such as ISD, instructional 
strategies, media selection, message design, and learner characteristics. 

When instructional design theory begin? Did it start with Finn's arguments (1960) for the integration of 
general systems concepts in the field traditionally known as audiovisual communications? Perhaps, but Finn was 
publishing ideas that grew out of the systems approach to training used by the military during World War II. Did 
design theory begin instead with Skinner's 1956 article on "The Science of Learning and the Art of Teaching" which 
led to programmed instruction and the marriage of ideas of psychology and instruction? DeVaney and Butler (1996) 
suggest otherwise. They argue that texts produced during the 1930s and 40s provided the base for the field of 
instructional technology and for instructional design. 

A different type of text, however, emerges in the late 30s and continues through the 40s. It is a text that 
attempts to ground audiovisual instruction in a learning theory and describe the manner in which theory 
suggests certain pedagogical practices. It is both theoretical and applied. The rhetoric of these texts engages 
the dominant educational discourse of the period, mid-30s to late 40s. It was a Deweyesque, child-centered, 
humanistic learning, and curricular theory, (p. 6) 

DeVaney and Butler report that Walt Wittich was writing about design elements in his 1944 dissertation on training 
manuals. 

To some extent instructional design theory began with the interest in making connections between concrete 
and abstract learning that is exemplified by the theory of John Dewey (Dale, 1967) and Charles Hoban Jr. (Hoban, 
Hoban, & Zisman, 1937). It was Edgar Dale who took this interest further by relating the concrete to abstract 
continuum to media decisions in his textbook, Audio-Visual Methods in Teaching. 

Although he was Jim Finn's mentor, Dale was solidly in the humanistic/communications tradition of the 
field of instructional technology (DeVaney and Butler, 1996). He was a pioneer who served as president of the 
Department of Visual Instruction of the National Education Association, now the Association for Educational 
Communications and Technology. He was a also leader in the fields of reading and journalism. His Dale-Chall 
Readability Formula is well known (Dale & Chall, 1948). For 36 years he published "The News Letter" which was 
mailed to 20,000 people around the world. Some of these now classic essays on communications in these 
newsletters are compiled in Can You Give the Public What It Wants (Dale, 1967). 

Dale was also my mentor, and I feel compelled to speak to misconceptions about his work because he has 
been labeled one of the realism theorists. He is said to have promulgated the theory that the more realistic the 
learning materials or experience the better. In fact, he said just the opposite. What he really said was that learning 
becomes more meaningful when abstract learning and concrete experience are related. He never said the more cues the 
better or the more realism the better. This is documented by his News Letter essays on "The Concrete and Abstract" 
and "Education as Conceptualizing" (Dale, 1967) and by his textbook Audio-Visual Methods in Teaching. 

Dale advanced the field theoretically through the Cone of Experience, which was a way to classify media 
experiences in relation to psychological dimensions of learning, specifically concrete and abstract. The Cone of 
Experience was introduced in 1946 in Audio-Visual Methods in Teaching (Dale, 1946, 1996). This text was 
published by Dry den Press, which produced seven editions within the first three years. By 1969 the validity of the 
cone approach to media selection was being questioned by Tosti and Ball (1969) who argued that it is not just the 
media it is how instruction delivered through the media is designed. They questioned whether media had unique 
characteristics because design can affect media characteristics. Nevertheless, the theory continued to be helpful to 
many, especially to those from other countries seeking a simple way to explain design theory. Dale's text was 
translated into Spanish and Japanese. 
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The historical importance of the Cone of Experience theory was its attempt to connect 
psychological/instructional theory and communications media. DeVaney and Butler believe that the cone was 
conflated, meaning two textual viewpoints were combined to create a new one. 

In this yearbook, we note a conflated discourse growing up around Dale-almost every author quotes Dale; 
and Brown and Vandermeer use Dale's Cone of Experience. Dale's own voice is conflated as he mixes the 
humanistic and experiential aspects of the child development curricular and learning movement with the 
sequential and hierarchical structure of task analysis proponents such as Charters'. (He worked on the 
Winnetka Plan and for Charters in the Payne Fund Study.) Dale’s Cone of Experience stands as an example 
of this mixture. It is at once experiential and hierarchical in its listing of experiential events. And, while 
offering an intriguing and popular model, it was based on conflicting theoretical assumptions, (p. 24) 

Dale would have loved the Latin derivation of the term "conflate," but he probably would have argued he was 
integrating ideas in to a whole instead of fusing them. 

The Cone of Experience presents a list of media on a continuum of direct, purposeful experiences to indirect 
experiences and a continuum of concrete to abstract learning. Today, it is easy to point out that films often provide 
experience that is as concrete and real as it is abstract and symbolic. In 1946, however, the capability and use of 
some media differed. However, a direct experience differed from a vicarious experience then and still does today. Dale 
believed that the cone turned on its side illustrated a spiral curriculum. 

Dale describes the Cone of Experience as a metaphor for concept development. As a model of 
conceptualization, it suggests that we have "sense-rich, purposeful, first-hand experiences" as participants. "As we 
move up the cone, the bands of experience become more abstracted" and we become spectators. "Finally, at the top 
of the cone we have the written name of the concept itself. It is an abstraction, a generalization. Its meaning depends 
upon what it stands for in the mind of the individual who reads or hears it." (Dale, 1956, p. 3) The cone, therefore, is 
tied to one type of learning, concept learning. 

Dale's other writings reflect remarkable foresight in many areas. He argued for the library-media movement 
by speaking of a "cafeteria for learning" and tempered the enthusiasm for programmed instruction and experimental 
research in our field with common sense observations. He edited the National Society for the Study of Education 
(NSSE) yearbook on "Mass Media and Education;" yet he was always a voice for interpersonal communication being 
stronger than mass media and necessary for "sharing in a mood of mutuality" (his definition of communication). In 
doing so, he presaged the two-step flow models of communication exemplified by diffusion theory. His chapter in 
the NSSE yearbook on programmed instruction presented many of the arguments against linear instruction that the 
cognitive science and constructivist paradigms would raise. He wrote about the need for rich "learning environments" 
throughout his career even after he retired. Dale did not justify design using the concept of "efficiency." He justified 
it as offering equity and individualization. He preceded formative evaluation theory; yet he developed the readability 
scale and other techniques to formatively evaluate the World Book Encyclopedia and health education materials. Most 
of his ideas have been brought to fruition with time and the growth of the field. 

However, the promise of his most important theory, the Cone of Experience, as a way to relate 
instructional psychology and communications technology is unrealized. While the direct to vicarious and purely 
symbolic experience continuum is still valid, the cone is dated in its description of media. The cone was introduced 
in a different period when there were fewer theories and those that existed were used more uniformly. Today, there is 
no widely accepted theory which follows in the tradition of the Cone of Experience. The result is a gap in 
instructional design theory which will be addressed next. 

ISD models synthesize the elements required for good instruction into a whole. Thus, content, objectives, 
assessment and instructional strategies, and delivery systems become integrated through systems and instructional 
theory. When one explains theory related to these steps in ISD, it is easy to show one theoretical thread that 
connects task analysis, objectives, assessment, and instructional strategy. This theoretical thread is types of learning. 
We identify types of learning during task analysis and use this information during instructional analysis. We base 
assessment strategy decisions on types of learning and objectives on assessment. Instructional strategy differs 
depending on the type of learning desired. Although this theoretical thread dominates, it is difficult to relate types of 
learning to selection of delivery systems. 

There may be other bases for connections between ISD steps and the delivery systems selection step. 
Certainly, the systems concept already provides one link. However, the link does not extend to 
instructional/psychological principles. There have been attempts to relate psychology and systems theory to 
communications media theory, but they have not been applied as theoretical links or procedures. 
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The descriptions of the technologies in the development domain in AECT's 1994 definition of the field 
(Seels & Richey, 1994) relates technologies to affinity for learning paradigms and linearity or non-linearity. 
Learning paradigms can affect steps in ISD other than delivery systems. For example, you can have a constructivist 
or a behaviorist approach to instruction for memorizing the Gettysburg Address. If you want someone to apply the 
writing style of the Gettysburg Address, you will have different objectives and assessment than you would have for 
memorizing the Gettysburg Address, but what objectives and assessment may still be influenced by a learning 
paradigm. 

Media selection models, such as Reiser and Gagn6's (1983) which incorporates types of learning as criteria 
for candidate media, are another basis for relating delivery system selection to other steps in ISD. Unfortunately, 
their procedure is tedious to apply and lacks the simplicity of appeal of theories such as learning paradigms or the 
Cone of Experience; nor is this theory clearly linked to conceptual theory. 

We are left with a theoretical gap between delivery system or media theory and instructional strategy theory 
and other steps in ISD. This gap is inconsistent with the promise of Dale's Cone of Experience and the dreams of 
ISD model makers. At the same time there are complaints that models simply do not reflect reality. The reality is 
that delivery system selection and development is equally important as other design steps and therefore, needs 
conceptual theory in addition to procedural theory. 

Are there new directions in theory worth pursuing in order to close the gap? Yes. Several. Here are some: 

• more specialized models 

• new ways to test models. 

• models which combine design, development, and evaluation 

• message design theory which relates types of learning and other variables 
I will address these directions next. 

Gros, Elen, Kerres, Merrienboer, and Spector have an article in the January 1997 issue of Educational 
Technology on "Instructional Design and the Authoring of Multimedia and Hypermedia Systems: Does a Marriage 
Make Sense." The article proposes that the relationship between ISD and authoring which links theory and practice 
be used to develop new ISD models. Furthermore, the authors suggest that such models can be made specific to 
types of learning. They recommend training to help designers translate general models to specific situations. The 
article states that in the context of the authoring of courseware: 

We have already argued that there are a variety of reasons for the apparent lack of integration of instructional 
design theories and practices. We listed five factors which make this integration difficult: (1) a mismatch 
between various ID models and the way developers actually work; (2) proliferation of ID models; (3) lack of 
specificity and the linearity of ID models; (4) absence of validation studies; and (5) the apparent linear 
character of ID models, (p. 54) 

The authors are calling for more powerful models that recognize both the design and the authoring stages and more 
rigorous and independent evaluation of products including objective examination of the ID process used by 
developers. They believe that designers and developers must combine a better conceptual grasp of ID with support 
tools for ID and that using tools without better conceptual theory will not be helpful. The authors seem to decree 
more general models while requesting new models that connect the design process with the concrete situation. It is 
interesting that one cry often heard in the field "No new models!" may be wrong. Perhaps we have too few models or 
inadequate training on applying and adapting models. 

In an article on "Instructional Design of Interactive Multimedia: A Cultural Critique" which appeared in 
Educational Technology Research and Theory, Lyn Henderson (1996) argues that we need ID models which allow for 
multiple paradigms to meet multicultural needs. Henderson contends that multiple paradigm models would allow us 
to relate design needs and development solutions more clearly. The article recommends new approaches to evaluation 
that would relate content decisions to navigation decisions. 

These two examples substantiate need for more complex models which relate design, development, and 
evaluation sufficiently. This may be one way to realize the promise of the "Cone of Experience" link between media 
and instruction. Several recent models combine the step of design and development including Willis' R2D2 model 
(1995) and Seels and Glasgow's ISD Model 2 (Seels & Glasgow, in press). 

Another way to provide conceptual links between media and instructional theory may be the development of 
conceptual and procedural models in areas where design and development are strongly linked such as message design. 
Seels, Mowery, O'Rourke, Proviano, Rothenberger, Tannehill & Yasin, 1996) have developed a conceptual structure 
and procedure which relates message design to types of learning, media, and learning paradigms. This theory has been 





incorporated in a performance support system for message design (Seels, Mowery, O'Rourke, Rothenberger, 
Vasadevan, & Sibiya, 1997). 

The technique of rapid prototyping (Tripp & Bichelmeyer, 1990) where design and development and 
evaluation steps are overlapped is another approach which may lead to closer links between the steps in ISD and the 
media selection step. The danger of this approach is there is a tendency to overemphasize procedural theory and to 
ignore design at the curriculum or macro level. 

It is useful to ask "Why has does this theoretical gap still exist after so many years"? One answer might be 
that we have more procedural than conceptual theory. Another might be a general feeling that if we adopt one 
perspective or paradigm we have to reject others. It may be that types of learning and media have an affinity for a 
paradigm, but that it is a mistake to assume this means incompatibility with other paradigms. Probably however, 
we simply haven't found the way to theoretically link delivery systems with other ISD steps because it is difficult to 
do so. However, to paraphrase a familiar saying, what is hopeless the first decade, is impossible the second decade, 
becomes difficult the third decade, and probable the fourth. We are, therefore, overdue to solve this problem. 
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Abstract 

This study was designed as an initial investigation into the construct validity of mental models. The 
content area selected was the use of spreadsheets. The intent was to analyze the correlation between verbal 
information and concept learning with Pathfinder measures of mental models acquisition. Although exploratory 
procedures were used for the factor analysis, the authors entered the analyses with research-based hypotheses. The 
expectation was for three distinct factors to emerge: one describing declarative knowledge, one conceptual knowledge 
and one describing structural knowledge. The data actually supported two factors, with structural knowledge 
variables loading on one factor and declarative knowledge on the second factor. Concept knowledge loaded on both 
factors ; this finding is discussed in the conclusions. 

Introduction 

A mental model is a knowledge structure, composed of concepts and the relations between them (Jonassen, 
Beissner & Yacci, 1993; Shavelson, 1974). Mental models are distinct from declarative and procedural knowledge 
(Jonassen & Tessmer, 1996). Anderson (1983) postulated the first stage of the development of expertise to be one of 
declarative encoding (distinct from procedural skills), where the subject is committing facts to memory. Declarative 
knowledge provides a base upon which subsequent learning can build relationships. The distinction between 
declarative knowledge and procedural skills is also made in R. Gagn6’s (1985) learning outcome taxonomy. Mental 
models, however, go beyond semantic relationships and skills acquisition. Mental models are knowledge structures 
that contain varied intellectual skills and knowledge. Jonnasen and Tessmer (1966) in addressing the distinctions, 
maintain: 

A mental model then contain three kinds of interconnected knowledge: knowing that (declarative), 
knowing how (skills), and knowing why (causal principles or functions). Conditional knowledge 
(knowing when) may also be a part of a mental model (p. 19). 

(A glossary of terms associated with structural knowledge, construct validity and factor analysis is contained 
in Appendix A.) 

There are at least three important implications of mental model research that should be considered by 
instructional designers. First, learners form mental models, whether the designer takes that fact into account or not, 
and inaccurate models can impede learning (Carroll & Thomas, 1982; Rouse & Morris, 1986; Norman, 1983). 
Second, troubleshooting can be facilitated through construction of mental models (Rouse & Morris, 1986; Gentner 
& Gentner, 1983). Similarly, structured knowledge has been found to be a powerful predictor of the ability to apply 
content knowledge (Gomez, Hadfield, & Housner, (1996). Finally, research on expert knowledge representation 
indicates that structural knowledge is essential to expert performance (Chi & Glaser, 1984; Larkin, McDermott, 
Simon, & Simon, 1980). Tardieu, Erlich, & Gyselinck, (1992), for example, found no difference between expert 
and novices at the prepositional level, but significant differences at the mental model level. Although the expert 
must have a sufficient knowledge base to draw upon, if it is not expertly structured, it is of little advantage. 

Construct History 

In recent years, several learning taxonomies have posited mental models as a learning outcome distinct 
from traditional outcomes of problem solving, concepts, and rules (Jonassen & Tessmer, 1996; Royer, Ciscero & 
Carlo, 1993). Before any construct becomes established as an independent learning outcome however, it should first 
be validated. As Rouse & Morris indicate, “...using a construct such as mental models results in requirements to 
define and illustrate the existence of such mechanisms (1986, p. 349).“ While the mental models construct becomes 
increasingly important, it yet remains unvalidated. Kraiger & Wenzel (in press) have developed an elaborate 
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construct validity process for team mental models, but it is narrowly adapted to team performance, and cannot 
substitute for validation of the broader construct. Thus, the mental models construct remains unvalidated. 

The evolution of the psychology of learning from a behaviorist to cognitive science has been in progress 
for at least several decades. The theoretical base in support of structural knowledge can be traced back to the early 
work of Bruner (Bruner, Goodnow, & Austin, 1956; Bruner, 1971); Ausubel (1960, 1968); Collins & Quillian 
(1969); Anderson (1974a; 1976; 1983; Anderson & Pirolli, 1984); and Rumelhart & Ortony (1977). Johnson-Laird 
reported that mental models as theoretical entities emerged as he wrestled with inference generation. He found the 
construct superior to other semantic representations for explaining meaning, comprehension and discourse (1983, p. 
397). In general, these authors agree that memory structures are acquired by the learner. Mental models are a type of 
memory structure, distinct from skills, declarative or conceptual structures (Jonassen & Tessmer, in press). 

Cognitive Importance 

Building on the theory base that information in memory is encoded in and retrieved from a structure that 
preserves meaning, theorists posit mental models as powerful engines for higher order cognitive processes. The 
mental model enables the learner to solve problems, generate inferences, and make predictions about the system that 
is modeled (Johnson-Laird, 1981; Wilson & Rutherford, 1989). Figure 1 models some of those functions. 



Figure 1. Purpose of Mental Models. 



DESCRIBING 

EXPLAINING 

PREDICTING 




-►WHY A SYSTEM EXISTS 
BOW A SYSTEM OPERATES 
■WHAT A SYSTEM IS DOING 
-►WHAT A SYSTEM LOOKS LIKE 



(Rouse & Morris, 1986, p. 349) 



Rouse and Morris’ work with troubleshooting and mental models would suggest another function 

of mental models learning: prediction what is wrong with a system (diagnosing). 

Williams, Hollan, & Stevens (1983) offer a helpful operational definition of mental 
models. They see mental models as containing a powerful interpreter, capable of being used as an inference engine 
for conducting the functions depicted above, as well as delivering explanations and justifications, and serving as a 
mnemonic device for keeping the throttle within easy reach. 

The Need for Construct Validity 

In a recent special issue of Educational Psychologist on the Role of Knowledge in Learning and Instruction, 
guest editor Alexander (1996) considered the history of cognitive research. She lauded researchers who investigated 
prior knowledge, the organization and structure of knowledge and the differentiation of the novice learner and experts. 
However, she cites Pintrich (1994) in concluding that “Although the notion of knowledge was central in early 
cognitive research, the construct was not often systematically defined or clearly operationalized” (p. 91). Designers 
researched learning outcomes and utilized them in developing instruction, yet the outcomes may not have been 
sufficiently validated by construct validity research. In short, mental models constructs are not so much invalid as 
“un valid”. Their distinction from other outcomes has not been researched. 

In a literature review, the authors found 202 hits on the ERIC database (1966-present) for “mental models,” 
but the Boolean addition of “construct validity” to the term produced zero hits. The PsychLit database returned 
similar results with no construct validity articles. Perhaps more troubling was an ERIC search of “concept 
learning,” a more recognized and traditional learning outcome. Of the 535 hits for that term, only one remained 
when “construct validity” was added. A search for “concept and construct validity” produced 105 hits, many of which 
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had to do with self- concept. The small number of construct validity studies dealt with only a few domains, notably 
Biology. In short, construct validity is conspicuously lacking in structured knowledge research. 

Campbell & Fiske (1959) wrote the classic and enduring treatise on construct validity. It 
calls for convergent and discriminant validity of the construct across differing measures and 
distinctly different constructs in a multitrait-multimethod matrix (MTMM). Although most 
researchers today prefer factor analysis over the rigorous MTMM (Meier, 1994), the general principles of the matrix 
approach are timeless. In a review of construct validity three decades after its debut, Cronbach (1989) succinctly 
summarized its methodology: 

Instruments supposedly measuring the same variable should correlate. . . 

If constructs are distinct, their measures ought to rank persons differently, or in some other way to 

give distinctive reports (p. 153). 

Messick (1980) describes construct validity as “a process of marshaling evidence” in 
support of the inferred meaning of the data set in the empirical relationships across measures. He uses the term trait 
validity to describe convergence of results on differing tests of a trait (construct) and nomological validity for the fit 
between theoretically divergent constructs and the data set (p. 1016). Cronbach and Meehl (1955) introduced the 
concept of a nomological (Greek, “legal”) network. It would include “the theoretical framework” for the construct, 
an “empirical framework” for measuring it, and the relationships between the two frameworks (Trokim, 1996, p. 
12 ). 

The mental models validation task, then, is to measure learning outcomes with different instruments and 
demonstrate the reality of mental models by convergence and divergence of the measures of the constructs in 
accordance with a hypothesized theoretical framework. The reality of the mental models construct is difficult to 
validate, because theoretically a mental model may contain concepts or verbal information as its model components. 
Concepts may be a necessary component of a mental model; declarative knowledge may not. However, a series of 
studies using convergent and divergent validity will marshal the necessary evidence to establish the credence of the 
mental model construct as a distinct learning outcome. This study is a first step in that direction. 

Purpose 

This study investigated the convergent and discriminant validity of the mental models construct as 
indications of a distinct learning outcome. To accomplish this, the study was initiated to consider the relationship 
between mental models, declarative knowledge, concept learning, problem solving and troubleshooting performance. 
Although this was an exploratory, first look into the subject, hypotheses were formulated that the mental model 
“scores” will 

1 . factor load with concept learning scores (convergent validity with a metal models component); 

2. not load with declarative knowledge measures (discriminant validity from a related outcome); 

3. load with the dependent variable of problem solving performance (predictive validity of mental 
models acquisition); and that 

4. the hypothesized structure of the exploratory factor analysis is for SPSS to identify three 
factors: one describing declarative knowledge, one conceptual knowledge and one describing 
structural knowledge. 

Method 

Subjects and Experts 

The subjects were a class of junior-level Operations Management (MGT 396) students in a public 
university in the southeast. A prerequisite for the course was completion of an introductory course on spreadsheets. 
The class was made up of 25 students, 21 of whom were present on the day of testing: 9 females and 12 males. 
During MGT 396, students use spreadsheets as a tool in performing management tasks, and not as an end in 
themselves. 

Three subject matter experts were employed to construct the testing instruments, the course instructor and 
two other college instructors. A Pathfinder network was drawn from each expert for comparison with the students’ 
networks. The instructor was a full professor in the College of Business, and holds a Ph.D. in Engineering. The 
second expert taught business application software and introductory computer programming at the university, and 
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has an M.S. in Computer Science. A third expert taught business application software at a local junior college and 
has an MBA. 

Design 

The Delphi technique ( Linstone & Turoff, 1975) was used to identify the concepts to be compared in a 
Pathfinder network and to generate a concept quiz. The technique was valuable because the experts teach very 
different student populations, and use spreadsheets for different instructional purposes (e.g., for production forecasting 
versus storage of financial data). Using the evolutionary Delphi process with a diverse base of expertise helped to 
identify concepts that epitomize spreadsheets. Sixteen core concepts were consequently selected by the experts. 

The final list of concepts was used as the terms for a Pathfinder network exercise using PCKNOT, an 
instrument used to assess structured learning attainment (Schvaneveldt, Durso, & Dearholt, 1989). Ten of the 16 
concepts were also used in a concept learning quiz in which the students identified examples of the concepts from a 
list of 14 choices. A 20 question verbal information quiz was also developed from a spreadsheet text, and was 
modified by the experts. It consisted of multiple choice questions concerning spreadsheet terms, inputs and outputs. 

Instruments 

Pathfinder Associative Network . Pathfinder derives network structures from proximity data drawn from 
subject responses to pairwise comparisons of conceptual terms (Shvaneveldt, 1990). Pathfinder networks have been 
successfully used to capture changes in mental models as a function of learning (Gonzalvo, Canas, Bajo, 1994; 
Canas, Gonzalvo, & Bajo, 1992) and to capture novice-expert representational shifts (Kellogg & Breen, 1990; 
Goldsmith, Johnson, & Acton, 1991; Acton, Johnson, & Goldsmith, 1994; Schvaneveldt, Durso, Goldsmith, Bree, 
Cooke, & De Maio, 1985) through the use of similarity measures. In the present study, two instructors’ networks 
served as the referent structures. The third network was rejected due to a lower internal coherence score. 

The Figure 2 is an expert network constructed from concepts related to the history of psychology. It shows that the 
experts’ network is highly organized, structured around several key concepts which form “ node neighborhoods”. 
(The length of the links connecting concepts is not part of the Pathfinder algorithm, and is not subject to 
interpretation.) The study that produced the network, like those above, used similarity as a measure of congruence 
between novices’ and experts’ structured knowledge. 
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Figure 2. Expert Pathfinder network. 
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Coherence is another important statistic produced by Pathfinder networks and measures the consistency of 
the pairwise comparisons made by a subject. Coherence is a correlation between the proximity between a pair of 
items and the relatedness of those items with all others in the set- the “interconnectedness of the knowledge 
representation*’ (Jonassen & Grabowsi, 1993). Higher correlations between the original proximity and the 
relatedness inferred from the indirect measures are indicative of expertise (Schvaneveldt, 1990). One might say that 
coherence is a plumb line held up against the subject’s knowledge structure. Relatedness ratings for a given pair of 
concepts should “square” with the all of the other comparisons, individually and collectively. 

Declarative knowledge q uiz. A set of 22 concepts and a 15 multiple-choice questions were extracted from 
self-help questions in a spreadsheet textbook (Bidgoli, 1993). The textbook offered no reliability or validity 
coefficients for the questions, and none were compiled from the small data set at hand. Sample questions from the 
final 20-question quiz can be found in Appendix B. 

Concept-matching quiz. The literature supports concept identification as a valid means of assessing concept 
knowledge (Gagn6, Briggs, & Wager, 1974). One method of doing so, is in a matching format. The subjects were 
asked to match 10 spreadsheet concepts to a list of 14 choices. An evaluation of this assessment will be found under 
the Discussion section. Again, this is an unvalididated instrument. The matching quiz can be found in Appendix C. 

Variables 

The variables included in a factor analysis must be representative of attributes of the construct(s) undo- 
consideration. It is tempting to include a large number of variables and observe what emerges from the analysis. 
There are at least three reasons for not doing so. First, as discussed later, supportable factor identification hinges on 
a reasoned theoretical basis for the construct. Second, a parsimonious set of variables will decrease likelihood of 
chance significance being observed. For example, a study with 30 variables will produce 435 correlations. At a 
significance level of .05, there could be 20 relationships deemed significant just by chance (Hair, Anderson, Tatham, 



& Black, 1995). Finally, since statisticians recommend a subject-to-variable ratio between five- and ten-to-one (Hair 
et al., Crocker, 1986), the number of subjects needed for a study can be kept manageable, usually a consideration in 
social science contexts. 

Since the object of factor analysis is to reduce some set of variables to a smaller number of factors which 
adequately represent the original set (see for example, Kim & Mueller, 1978), the variables must have a substantial 
degree of covariation, known as communality. The variables in this study exhibit a moderately high degree of 
covariance. The variables invested were: 

1) the results of the verbal information quiz; 

2) results of the concept matching quiz; 

3) internal coherence of the students’ Pathfinder networks; 

4) similarity between students’ pathfinder networks and their instructor’s network; and 

5) similarity with a second instructor’s network. 

Exploratory Factor Analysis 

Factor analysis is a multivariate statistical method used to uncover the underlying structure in a data set. 
This is accomplished by identifying a small number of factors (latent variables) that are representative of the 
relationships in a larger set of constituent variables. The question is not one of relationships between dependent and 
independent variables, but of identifying factors that are not directly observable from a set of variables. The 
distinguishing feature of factor analysis is the assumption that the covariation amongst the variables is due to an 
underlying structure of common factors (Kim & Mueller, 1978). The study sought to determine whether the 
knowledge variables will cluster around factors that suggest the existence and measurability of mental models. 

In confirmatory factor analysis, we enter the analysis with an a priori structure for the data. CFA seeks to 
answer the question, “Do the data fit the hypothesized model?” and acts as a deductive reasoning tool for theory 
testing. Although the authors had formulated hypotheses about emergent factors and relationships among the 
variables, this was very much an exploratory factor analysis. In EFA, the researcher allows the latent structure to 
emerge from the data set a posteriori. EFA serves as an inductive reasoning tool for theory building (Bryant & 

Yamold, 1995, p. 109). The number of expected factors was not imposed upon the factor analysis. Rather, 

Eigenvalues of 1 .0 or greater (latent root criterion), and a Scree plot were used. An Eigenvalue (the column sum of 
squared loadings for a factor) represents the amount of variance of the original set of variables accounted for by a 
given factor. The Scree test criterion seeks to identify the point at which unique variance begins to exceed the 
common variance in the structure. Latent roots are plotted against the number of factors. The point at which the 
slope begins to straighten out represents to maximum number of factors to extract (Hair et al, 1995). Four steps 
were used in the factor analysis: 

1. compute the correlation matrix for the variables; 

2. extract the factors from the correlation matrix; 

3. rotate the factors to make them more interpretable; and 

4. analyze and name the factors. 

In factor rotation, the reference axes of the factors are rotated, seeking a theoretically more meaningful 
representation of the original variables. Varimax, an orthogonal rotation procedure, was used in the study. 
Orthogonal rotation maintains the factors at a 90 degree orientation; that is, the factors are uncorrelated with each 
other. That is reasonable, in that the study seeks to identify unique factors that represent distinct constructs. Figures 
3 and 4 are an oversimplification, but graphically represent the theory and process of rotation. 
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Figure 3. Unrotated variables and factors. 
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Figure 4. Rotated variables and factors. 




“Simple structure” would describe a factor matrix loading pattern where each variable loaded very heavily 
on one factor in the matrix, and very little or not at all on the others. An example might look like the factor matrix 
in Table 1. 



Table 1. Example of simple structure. 



Factor 1 Factor 2 Facto r 3 



Course GPA 


1.0 


0 


0 


Problem Solving Lab 


0 


0 


1.0 


Pathfinder Coherence 


0 


0 


1.0 


Pathfinder Similarity 


0 


0 


1.0 


Declarative Knowledge Test 


1.0 


0 


0 


Concept Test 


0 


1.0 


0 



In an imperfect analogy, the factor loadings can be thought of as correlation coefficients for the respective variables 
(Jaeger, 1990). That is, the results of the Declarative Knowledge Test would have perfect correlation with Factor 1 , 
and no relationship with the other two factors. In this study, we expect to find variable loadings that would indicate 
the reality of mental models. If, for example, Verbal Information in our study were to load heavily on the same 
factor as Pathfinder Internal Coherence, the existence of mental models would not be supported. 

It is the factor matrix produced in step three that must be analyzed in conjunction with the theory base for 
the constructs. The first three steps were simply a procedural function of the software program. The last step 
incorporates both art and science: a theory-based interpretation of the statistical results. 

Results 

Step 1: The Correlation Matrix. Table 2 present the correlation coefficients. An explanation of the 
variables may be found in the factor matrix under Measure. 



Table 2. Pearson correlations among the variables. 



COHERENCE 

CONCEPTS 

VERBINFO 

ZSMLRTY 

PSMLRTY 



COHERENCE 


CONCEPTS VERBINFO 


ZSMLRTY 


PSMLRTY 


.222 









-.007 


.511** 






.619*** 


.391* .051 




— 


.518** 


.164 


.405 


.61 7" 



n=14; *Correlation (1-tailed) is significant at the 0.10 level; ** the 0.05 level ; *** the 0.01 level . 



Verbal information (declarative knowledge) did not correlate with any of the mental models variables, but 
moderately correlated with concept identification (^.51; p=. 03). Concept identification was moderately correlated 
with student-course instructor similarity (^.39), and was statistically significant at the .10 level. This is an 
acceptable alpha level for exploratory studies (Borg, & Gall, 1989). Coherence was moderately and significantly 
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correlated with similarity ( ^=.62; p-. 01). The correlation between coherence and concepts was .222; with verbal 
information, -.01. Neither relationship was statistically significant. 

Step 2: Factor Extraction. Two methods were used to determine the number of factors to extract: 
Eigenvalues of 1 .0 or greater (Appendix D), and a Scree plot. The knee of the Scree plot (Figure 5) falls at the three 
factor point, but the third factor accounts for a relatively small amount of the total variance, making it of little use 
as factor in the study. 

Figure 5. Scree plot (SPSS output). 



Factor Scree Plot 




Factor Number 



Step 3: Factor Rotation. Varimax rotation with Kaiser normalization was used, and converged in 
three iterations. Two factors were extracted. The factor loading matrix is presented in Table 3. 



Table 3. Factor loading matrix. 







Factor 1 


Factor 2 


% Var 


Measures 


1 


COHERENCE 


.833 


.034 


47.2 


1 Internal consistency of 
Pathfinder models. 


2 


CONCEPTS 


.270 


.839 


27.9 


2 Score on a concept quiz. 


3 


PSMLRTY 


.829 


.000 


10.3 


3 Similarity with a second 

instructor’s Pathfinder network. 


4 


VERBINFO 


-096 


.888 


09.0 


4 Score on verbal info quiz 


5 


ZSMLRT 


.871 


.191 


05.7 


5 Similarity with course 
instructor’s Pathfinder 
network. 
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Step 4: Analysis and Identification. The factor matrix above approaches simple structure, with each 
variable loading high on one factor and low on the other, with the exception of the scores on the concept knowledge 
quiz. That variable, though loading highly with verbal information on Factor 2, also loads moderately with the 
more structural variables on Factor 1 (Coherence, Zsimilarity, and Psimilarity). Coherence accounted for 47.2 
percent of the total variance amongst the variables; scores on the concept quiz accounted for almost 28 percent. 
Communality (the amount of variance of a given variable shares with the other variables) was moderate to high, 
ranging from .69 to .80, with a mean of .75 (Appendix D). That is, about 75 percent of the variance in the five 
variables selected is held in common; this is a fertile environment for factor analysis. Using Varimax rotation, 
SPSS extracted two factors. Based on the mental models literature, emergent Factor 1 is tentatively labeled Mental 
Models, and is composed of network coherence and similarity measures. Factor 2 is tentatively labeled Semantic 
Knowledge, and is comprised of verbal information and concept measures. 

Discussion 

The extracted factors accounted for about 75 percent of the variance within each of the variables, indicating 
that the two-factor model was a good fit for the five variables used. Some of the results supported the research and 
hypotheses, some did not. The most important finding was that all three purely structural variables (coherence and 
both similarity variables) loaded almost exclusively on one factor and verbal (declarative) knowledge loaded 
exclusively on the other. These results support the discriminant validity of the mental models construct. Moreover, 
network coherence (the structural “plumb line”) accounted for almost half of the total variance amongst the variables. 

As expected, the students’ mental model internal coherence correlated much more positively with concepts 
than with verbal information, although the correlation was not large. This supports the proposition that mental 
models are comprised of conceptual relationships more than declarative knowledge (Jonnasen & Tessmer, 1996). 
However, verbal information correlated positively with concept identification, and the latter loaded more heavily on 
the “semantic knowledge” factor, indicating that these outcomes are not as discrete as instructional design literature 
hold them to be. Part of the overlap can be explained by prerequisite knowledge: verbal information of concept 
attributes can facilitate concept learning. It is interesting that in the unrotated factor matrix (Appendix D), Concepts 
loaded .57 on Factor 1 and .68 on Factor 2. This is obviously a much more even spread of its variance than the .27 
to .84 ratio in the rotated matrix. The loading of the concept learning variable when a rigorous test of concept 
understanding is used will be instructive. 

It was noted above that the coherence of the students’ Pathfinder networks accounted for nearly half of the 
total variance amongst the knowledge variables. Some discussion of the distribution of the remaining variance is in 
order. The subjects were attending their third class meeting of Operational Management. Their instructor had a rich, 
well developed knowledge base in the use of spreadsheets in novel and innovative contexts. Each student had taken a 
prerequisite course, where spreadsheets were presented as the course content in a simple, mechanistic manner. The 
second instructor taught such an introductory course in spreadsheets. The second instructor’s approach to 
spreadsheets was more “in line” with the students’ experience with spreadsheets. Had the study continued, the course 
instructor’s Pathfinder network would be hypothesized to correlate increasingly more with the students’. 

The concept learning outcome unexpectedly loaded with the “semantic” factor more heavily than the 
‘structured” factor. Using a matching exercise for the identification of concepts was apparently not a rigorous 
enough test. Memorized information would have been sufficient to correctly identify the concepts. The quiz did not 
require a deep understanding of the concepts on the part of the subjects. Tessmer, Wilson & Driscoll (1990) offer 
alternative recommendations for the measurement of concept learning that go beyond simple example identification: 

1) using the concept in conversation, writing and argumentation (Brown et al., 1989); 

2 ) simulating or role-playing the concept (Tessmer, Jonassen & Caverly, 1989); and 

3 ) making judgments or criticisms on the basis of the concept (p. 48). 

It is anticipated that if a more demanding measurement instrument was used to assess concept knowledge, 
the results would load on the “structural knowledge” factor, and not correlate as strongly with verbal 
(declarative) knowledge. 

This initial investigation has several limitations. The study was terminated after the data set considered in 
this paper was collected, which caused some of the limitations. Others have to do with the instruments and 
methodology used. The limitations are : 

1. Subject mortality. Five students chose the same level of comparison for each pair of concepts during 
the Pathfinder exercise. There was no payoff for them to take the time to make the large number of comparisons. 
Pairwise comparison measures can be tedious tasks that tempt the subject to hasten their comparison ratings. To the 
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degree that there was a correlation between the population that failed to perform the comparisons and any of the 
variables, bias was introduced. 

2. Generalizability. Because neither random selection or assignment was used, the external validity is in 

question. 

3. Missing variables. Because the study was terminated early, potentially important variables were 
missing from the matrix. When factored in, they may have had significant impact on the resultant factor matrix. 

4. Insufficient n. The number of subjects (14) was insufficient for a study of this nature. 

5. Unvalidated instruments. The verbal information and concept tests were unvalidated. Without a valid 
instrument, it is difficult to maintain that a hypothesized construct has actually been measured. 

6. Convergent validity. Only Pathfinder networks were used to assess structured knowledge. 

Recommendations 

Some final recommendations should be considered for future research into construct validity for mental 
models. First, provide an incentive for subject participation in the Pathfinder data collection. Next, use an n equal 
to five-to-ten times the number of variables being factored (Crocker, 1986), ideally 50 in this study. Third, test 
concept knowledge by using rigorous measurement of inferential abilities of the students. Fourth, use only valid 
instruments to measure the variables. Finally, use multiple methods and measures. Perhaps card sorts and 
interviews with students in addition to Pathfinder networks to assess mental model attainment and similarity with 
experts. The alternative methods for concept knowledge assessment mentioned earlier could be used. Also, multiple 
regression could be used to measure the predictive power of mental models upon problem solving. 

Conclusions 

Instructional designers have long been involved in the research and use of learning outcomes, but have not 
always been as quick to validate those outcomes before incorporating them into taxonomies. This study was a first 
step towards construct validation of the mental models learning outcome. A great deal is left to be done. 

Cronbach’s (1984) view is that “Construct validation is a fluid, creative process. ..to develop an interpretation, 
persuade others of its soundness, and revise it as inadequacies are recognized (p. 149).” We are still in that creative 
process, but we have a tentative interpretation of the mental models construct. Now we need to move toward 
“soundness” through instruments and variables that have persuasive power. We close with Messick’s (1980) 
summary: 

Thus, the paradox that measures are needed to define constructs and constructs are needed to build 
measures is resolved, like all existential dilemmas in science, by a process of successive 
approximation (p. 1016). 

Future studies will continue the process, and move successively closer to a valid mental models construct. 



ERIC 



373 



382 



Appendix A 



Definitions 

Structured Knowledge Terms 

Structural knowledge describes an individual’s organization of ideas (knowledge structure) about different content 
domains. This knowledge is essential to understanding the content and the ability to apply it (Jonassen & 
Grabowski, 1993, p. 434). 

Propositions represent the atomic units of meaning and can be used to represent the meaning of sentences and 
pictures. The interconnections among propositions define a network (Anderson, 1990, p. 143). 

Certain groups of propositions group together in larger-order units called schemas. Here, the abstraction is from 
specific instances to generalizations about the category from which these instances come (Anderson, 1990, pp. 144, 
133). 

Mental models are the mechanisms whereby humans are able to generate descriptions of system purpose and form, 
explanations of system functioning and observed system states, and predictions of future system states (Rouse & 
Morris, 1986, p. 351). 

Pathfinder networks represent concepts as nodes and distances between concepts as links in a network. The 
Pathfinder algorithm takes proximity values as input and produces as output a network with shortest-path link 
distances (Acton, Johnson, & Goldsmith, 1994, p. 305). 

Construct Validity Terms 

The term construct is used. . .to refer to something that is not observable, but is literally constructed by the 
investigator to summarize or account for the regularities or relationships that he observes in behavior (Thorndike & 
Hagan, 1986, p. 70). 

Construct validity refers to the nature of the psychological construct or characteristic being measured by the 
instrument. How well does this construct explain differences in the behavior of individuals or their performance on 
certain tasks (Fraenkel & Wallen, 1996, p. 154). 

Factor Analysis Terms 

Communality is the amount of variance an original variable shares with all other variables included in the analysis. 
An Eigenvalue (or latent root) represents the amount of variance accounted for by a factor. 

A factor is a linear combination (variate) of the original variables. Factors also represent the underlying dimensions 
(constructs) that summarize or account for the original set of observed variables. 

Factor loadings are correlations between the original variables and the factors, and the key to understanding the nature 
of a particular factor. 

Factor rotation is the process of manipulating or adjusting the factor axes to achieve a simpler and pragmatically 
more meaningful factor solution (Hair, Anderson, Tatham, & Black, 1995, pp. 365-366) 
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Sample Questions — Declarative Knowledge Quiz 



Select the most correct answer for each question by writing the corresponding letter. 

1. The intersection of a row and column is called a 

a. cell 

b. pointer 

c. function 

d. formula 

e. none of the above 

2. Using a spreadsheet for what-if analysis, you can 

a. change one value and see the result on the rest of the worksheet 

b. change one value and create a new graph 

c. do both a and b 

d. change one graph and create a new value 

e. do none of the above 

12. Compared to /Range Format, /Worksheet Global Format has 

a. lower priority 

b. higher priority 

c. equal priority 

d. priority determined by the type of calculation 

e. none of the above 



15. T he most suitable graph for time series analysis is a (an) 

a. pie chart 

b. exploded pie chart 

c. any graph that uses symbols 

d. line graph 

e. none of the above 



16. The X range is used in 

a. pie charts 

b. exploded pie charts 

c. graphs that use symbols 

d. line graphs 

e. all graphs 
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Apppendix C 



Concept Matching Quiz 

Match the concept in Column A with the corresponding example in Column B. 
may be used once, more than once or not at all. 



COLUMN A 


COLUMN B 


21. 


Argument 


A. AB2-CD4 


ZL— 


Cell Address 


B. ACME.WK1 


22^_ 


File 


C. A5 





Formula 


D. {CALC} 





Graphics 


E. B1..C4 


2fL 


Label 


F. Champions. E 


2L 


Macro 


G. FEMALE 





Numeric Constant 


H. +Bl*Hl/2 





Range 


I. WYSIWYG 





What-if 


J. 3.1416 



K. @MaxAl..B3 

L. @I^ngth(‘Title’’&”Sub-Title”) 

M. .PIC 

N. 9.00 

Thanks for your help with this project. 




q 




Examples 
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Appendix D 



SPSS Factor Analysis Outputs 



Correlation Matrix: 

COHERE CONCEPTS PSMLRTY VERBINFO ZSMLRTY 



COHERE 


1.00000 








CONCEPTS 


.22243 1.00000 








PSMLRTY 


.51848 


.16370 


1.00000 




VERBINFO 


-.00733 .51050 


.00653 


1.00000 




ZSMLRTY 


.61867 


.39096 


.61673 .05086 


1.00000 



Determinant of Correlation Matrix = .216543 1 

Principal Components Analysis (PC) Initial Statistics: 

Variable Communality Factor Eigenvalue PctofVar Cum Pet 



COHERE 


1.00000 


1 


2.35804 


47.2 


47.2 


CONCEPTS 


1.00000 


2 


1.39304 


27.9 


75.0 


PSMLRTY 


1.00000 


3 


.51319 


10.3 


85.3 


VERBINFO 


1.00000 


4 


.45074 


9.0 


94.3 


ZSMLRTY 


1.00000 


5 


.28499 


5.7 


100.0 





Rotated Factor Matrix: 




Factor 1 


Factor 2 


COHERE 


.83323 


.03386 


CONCEPTS 


.27047 


.83910 


PSMLRTY 


.82899 


-.00042 


VERBINFO 


-.09561 


.88757 


ZSMLRTY 


.87051 


.19105 



VARIMAX converged in 3 iterations. 





Unrotated Factor Matrix 

Factor 1 Factor 2 


COHERE 


.78480 -.28197 


CONCEPTS 


.56617 .67579 


PSMLRTY 


.76798 -.31214 


VERBINFO 


.24518 .85838 


ZSMLRTY 
Wrf 


.87845 -.15034 



Variable 


Communality 


Factor 


Final 

Eigenvalue 


Statistics: 
PctofVar Cum Pet 


COHERE 


69541 


1 


2.35804 


47.2 47.2 


CONCEPTS 


.77724 


2 


1.39304 


27.9 75.0 


PSMLRTY 

VERBINFO 

ZSMLRTY 


.68722 

.79692 

.79428 
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Multimedia Matrix: A Cognitive Strategy for Designers 

Annette C. Sherry 
University of Hawaii at Manoa 



Abstract 

This instructional development project evaluates the effect of a matrix-based strategy to assist multimedia 
authors in acquiring and applying principles for effective multimedia design. The Multimedia Matrix , based on the 
Park and Hannafin (1993) Twenty Principles and Implications for Interactive Multimedia design , displays a 
condensed version of those Principles . Space is provided for students to generate their own responses to the presence 
or absence of elements as they evaluate multimedia modules before and after they begin their own designs. 
Instruction followed Jacobson’s (1994) Theory-to-De sign framework. Formative evaluation of the project is based 
on qualitative and quantitative data reflecting perceptions of graduate students , enrolled in a course containing a 
multimedia authoring component, toward this Matrix-based approach. Their perceptions toward the Matrix-based 
authoring instructional design and their and the evaluators’ analyses of the students’ subsequent multimedia 
productions are presented and analyzed in relation to the effectiveness of applying two versions of the Multimedia 
Matrix. Implications for future application and investigation of the Matrix is provided 

The increase in authoring environments for multimedia 3 design has not necessarily lead to increased quality 
in the resulting productions (Locatis, Ullmer, Car, Bernard, Le, Lo, & Williamson, 1992; Jones & Smith, 1992; 
Stanton & Baber, 1994); yet scant guidance is offered to designers to connect them with relevant research and 
learning theory (Ambrose, 1991). To provide guidance for multimedia designers concerned with developing quality 
modules, Park and Hannafin (1993), developed Principles and Implications for Interactive Multimedia Design based 
on twenty psychological, pedagogical, and technological foundations from research findings. For example, to apply 
their Principle 15 — “Learners become confused and disoriented when procedures are complex, insufficient, or 
inconsistent.” — they suggest that the designer should, “Provide clearly defined procedures for navigating within the 
system and accessing on-line support.” (Park & Hannafin, 199 p. 77-78). 

As novice and intermediate level designers attempt to incorporate these Principles into their multimedia 
productions, while simultaneously integrating technical knowledge of authoring programs and subject matter 
knowledge, they face challenging, higher order cognitive demands. In such situations, research in cognitive science 
and instructional technology, suggests that integrated, contextually-based instructional strategies be considered 
(Tennyson, Elmore, & Snyder, 1992). 

Many beginning hypermedia authors tend to be teachers who have subject matter expertise and little or no 
knowledge about technical aspects of multimedia authoring. No specific guides exist for introducing authoring 
concepts such as those encompassed by the Principles . An attempt prior to this investigation, to introduce the 
Principles using a broad-based approach of expository techniques of assigned reading and discussion of the Park and 
Hannafin article, a demonstration, a brief Principles checklist, and encouragement of incorporation of relevant 
principles into initial multimedia productions, resulted in modules that were judged by the both the students and the 
instructor to contain only some aspects of the Principles . The majority of the students, however, expressed concern 
over the challenges posed by trying to remember the Principles while working in the multimedia environments. 
(Sherry, 1995). Discussion with one of the authors of the Principles (R. Hannafin, personal communication, 
February 16, 1995), resulted in agreement about the potential usefulness of developing a visual guide, hereafter 
referred to as the Matrix. An instructional design was developed that included the use of a Matrix crafted to serve as a 
visual, cognitive reference with which students could generate specific responses as they worked in a contextually 
relevant setting. 

It is not surprising that research on the effectiveness of matrices to concepts, is mixed (Newell & Olejnik, 
1982; Rumelhart, 1980; Siebold, 1989; Spiegel & Barufaldi, 1994), given the great diversity of matrix designs and 



3 Multimedia — "the use of multiple formats for the presentation of information, including text, 
still or animated graphics, movie segments, video, and audio information. Computer-based interactive 
multimedia includes hypermedia and hypertext." (Tolhurst, 1995, p. 25). 
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usage reported in such studies. By employing a systematized approach to usage, combined with feedback from the 
participants, a critical element in effective evaluation (Reiser & Kegelmann, 1994), a Matrix-based instructional 
design was developed and refined through the testing of two versions of the Matrix developed to specifically guide 
users in applying the Principles . 

Method 

Participants 

Seventeen graduate level education majors, involved in multimedia development participated in providing 
preliminary data on the Matrix-based instructional design project. Technical abilities ranged from beginner to 
advanced with all having no prior knowledge of the Park and Hannafin (1993) Principles. 

Procedures 

Instruction was based on the seven steps of Jacobson’s Theory-to-Design framework (1994), that employs 
cognitive flexibility theory (CFT) to support the “development of flexible representation of knowledge that will help 
promote deep conceptual understanding and ability to adaptively use knowledge in new situations” 

(p. 146). The seven steps that were followed were: 

1) Employing rich cases and examples — An exact statement of the twenty Principles and an abbreviated 
version of implications for applying the Principles was developed and displayed on versions one and two of a Matrix 
created for student use. Blank cells were provided for making notations about key elements of effective multimedia 
authoring that could be identified during a review of multimedia modules. The Park and Hannafin article and five 
examples of student-produced multimedia modules were provided along with the first version of the Matrix during 
instructor demonstration and student analyses . 

2) Using multiple forms of knowledge representation — The articles, the Matrix v.l, the modules, and a 
talk-aloud approach were used to present the information in multiple forms. Students, working in self-selected 
dyads, chose three of the five multimedia modules to evaluate using the first Matrix as a guide. 

3) Linking abstract concepts to case examples — A variety of the abstract Principles were contained in the 
sample modules. 

4) Demonstrating conceptual complexities and irregularities — The modules that served as examples varied 
in the quality and quantity of complex features of multimedia Principles that they incorporated. 

5) Stressing the interrelated and weblike nature of knowledge — The Matrix was designed to be interrelated 
to the Park and Hannafin article. Version one was used when the students evaluated multimedia modules developed 
by others; version two was used when they evaluated their own modules. 

6) Encouraging knowledge assembly from different conceptual case sources — The students were required to 
analyze modules in writing, as well as to construct their own analysis of the effectiveness of the first Matrix. Their 
suggestions for change were incorporated in the second version of the Matrix. 

7) Promoting active learning — Throughout their authoring experience, students were actively engaged with 
analyzing and applying both versions of the Matrix, the article, and the modules and with constructing their versions 
of the Matrix approach. 

Data Analysis 

Quantitative and Qualitative Data 

Both quantitative and qualitative data were gathered to evaluate the overall effectiveness of the Matrix-based 
instructional design and the students’ ability to apply the Principles to their designs. Quantitative data consist of 
students’ mean scores on aspects of the usefulness of the multimedia production and of the Principles , as well as on 
the overall instructional design of the course. Quantitative data consisted of interval data from a written survey 
administered at the end of the course; the percentage of completed multimedia productions judged by the student 
authors as being worthwhile additions to the software preview center of the college; the degree of effectiveness that 
the students reflected toward the two versions of the Matrix as rated by two evaluators; and the percentage of students 
meeting criterion-based measures for presentations and reports that reflected attainment of declarative and procedural 
outcomes as judged by instructor ratings. Qualitative data are reported in an analysis and ranking of comments from 
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the students about both the Matrix and their modules in relation to application of principles for effective multimedia 
design and in the instructor’s overall evaluation of all components of the evaluation process. 

Formative Evaluation — Matrix 

Phase I. In Phase I students were asked to provide written evaluations of the first version of the Matrix in 
terms of its positive and negative effects. Their suggestions for changes to the guide were also sought. Two 
evaluators analyzed these responses to identify commonalties in the comments. The resulting themes were ranked 
according to the most frequently occurring items. Based on this analysis, changes were made to the first version of 
the Matrix. The revised Matrix v. 2, was made available to all the students during their subsequent multimedia 
authoring. 

Phase II. In Phase II, the students’ multimedia productions were evaluated by the instructor in relation to 
their ability to reflect the Principles in their productions. A minimum of ten of the twenty principles was set as the 
baseline for effective design for their productions. An option was provided for students to analyze their modules in 
writing in relation to Matrix v.2 and to provide a log of the time spent using the matrices. 

Students provided additional written responses in relation to the interactive multimedia component of the 
course by responding to three items on a Likert-like survey administered at the conclusion of the course. 

Results and Discussion 

This evaluation emphasized listening to student opinions, as this approach was determined to have the 
greatest potential for providing relevant data on ways to improve discrete elements of the multimedia module, as well 
as the overall design of the course (English & Reigeluth, 1996; Lohr, Ross, & Morrison, 1995). The Matrix-based 
course component was perceived by learners to effectively contribute to their intellectual curiosity (4.5), their 
satisfaction (4.5), and general perceived usefulness (4.5) as indicated by learner responses to a survey administered at 
the end of the course, that contained a Likert-like scale for responses that ranged from 1 (almost never or almost 
nothing) to 5 (almost always, a great deal). Of all the course components, multimedia production was perceived to 
be the most useful one. 

In analyzing the students’ perceptions about both versions of the Matrix, comments offered by more than 
25% of the students were reviewed. As shown in Table 1, within those parameters, the students provided more 
negative and change comments than positive ones for the first version of the Matrix. 

Based on these evaluations, changes were made to the text of the Matrix to clarify vocabulary and redundant 
information. A checklist format was adopted for greater consistency and ease of use. The response boxes underwent 
minor reformatting to systematize the overall design of the instrument. To clearly keep the relationship of the 
Matrix to its accompanying article, no additional items were added, although some students had made that request. 
The revised Matrix is depicted in Figure 1. 

The analysis of the revised Matrix, indicates a major change with comments in the positive category 
outnumbering the negative and change categories. According to students the new version does achieve its purpose of 
simplifying the application of the Principles for the authors. This comment was provided by 82% of the respondents 
and achieved the top ranking for all items. 

While the main negative comments that appeared for the first version did not occur for the second one, the 
shape of the response boxes was still seen as a problem. Although the newer version appears to support users’ needs 
to a greater extent than the initial one, these results should be interpreted with caution. The results of the student- 
based evaluations of Matrix v.2 are based on responses from eleven of the seventeen students. In keeping with the 
constructivist-based design of this part of the course, students could determine whether or not to use and evaluate 
Matrix v.2. Of the six students who did not elect this option, five had had negative reactions toward the first version. 
The effect of these students’ non participation in the final evaluation needs to be kept in mind to avoid interpreting 
the students’ second round of comments in an overly optimistic manner. 

When the students’ views of the Matrices are investigated in relation to their skills as authors — novices or 
more advanced designers — there is a similar pattern within the number of the types of responses each group of 
students offers in relation to positive, negative and change aspects. For example, novices provided more positive 
comments than advanced students and less when making negative responses and suggestions for change. It may be 
that the novices welcomed any type of support, while the advanced students were more discerning about the type of 
support they required. Novices were more apt to mention version one of the Matrix as contributing to guiding 
evaluators and supporting consistent feedback. The advanced students also recognized the contribution of guiding 
evaluators, but were more apt to mention support of consistent feedback and focusing attention. It could be that the 
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novices perceive consistency as requiring less mental effort when considering the Principle s, allowing them to 
concentrate on mastering the authoring program itself, an issue of less concern for the more advanced learners. 

By version two, as the novices had become more experienced in manipulating the authoring program, they 
cited simplifying the application of the Principles and/or convenience and an effective checklist format as their two 
most frequently mentioned items. The predominant response of the advanced students was the same for both items, 
but they also recognized the Matrix as a guide for designers by mentioning that aspect as frequently as they 
mentioned its checklist feature. As advanced students reported spending less time using the matrices (an average of 
one hour less with Matrix v.2 than the novices), the former students may have found more time for reflecting on the 
effect of the Matrix and, thus, come to this conclusion. 

Regarding negative comments and suggestions for change, fifty percent of the novices stated that Matrix v.l 
was inconsistent if used by many and had vocabulary problems. Only thirty-six percent of the advanced students 
recognized the issue of inconsistency, and as such, it appeared as the second most frequently appearing item for them 
in their negative comments. Their most frequently mentioned negative item was redundancy. Forty-five percent cited 
that issue. After using the second version, the novices showed a marked decline in their negative comments. None 
mentioned inconsistency as a problem and only twenty-five percent continued to cite the vocabulary problem. Issues 
of redundancy and requiring prior knowledge appeared at this stage for twenty-five percent of the novices. It may be 
that as the students became more confident with their developing authoring skills they became more positive. A 
similar change appeared with the advanced students as negative comments declined to only one. That one comment 
regarded the design of the response boxes with forty- three percent criticizing the shape. The appearance of that type of 
comment by a large percentage of the advanced students may be a result of their desire to have more space for their 
comments as these students were observed to be more apt to write remarks than were the novices. 

The change comments followed the pattern for the negative comments. Suggestions for change decreased 
between the two versions of the Matrix. The novices had requested that the initial matrix be reformatted to a 
checklist format and that a provision be added for overall scoring. As both these features were incorporated into the 
revised version, no comments were offered about these topics for Matrix v.2. The advanced students suggested that 
items other than those covered by the twenty Principles b e added to the Matrix after they worked with Matrix v.l. 
This request indicated their broader knowledge of other aspects of multimedia that were not addressed by Park and 
Hannafin. As the intent was to create an instructional tool that would support and enhance the Park and Hannafin 
article, no attempt was made to add items to Matrix v.2. This decision was explained to the students. Both the 
novices and the advanced students did request that Matrix v.2 be simplified and shortened to be less time consuming. 
Novices did report spending approximately one hour longer than the advanced students when working with the 
Matrix, with times reported as 2 1/2 hours for the former and 1 1/2 hours for the latter. Such amounts of time, do 
indicate a need to refine the instrument for greater efficiency. 

The results of the quantitative data in the end of the course survey have greater potential for reliably 
reporting the students’ view toward the overall instructional aspects of the multimedia component as all students 
provided responses. On a Likert-like scale that ranged from 1 (almost never or almost nothing) to 5 (almost always, 
a great deal), the learners indicated that the multimedia authoring component effectively contributed to their 
intellectual curiosity, their satisfaction, and general usefulness as indicated by a mean of 4.5 for each of the threee 
items. 

Ten modules were developed by the students, five by individuals and five by groups. All modules were 
judged by the instructor to address a minimum of ten of the principles. While this evaluation has the potential for 
instructor bias, the value of the modules was supported by the multimedia authors. Sixteen of the seventeen 
students judged their productions as worthwhile contributions to the Technology Learning Center of the college, a 
site that serves as a preview center for commercially- and student-produced educational software. Peer review and exit 
interviews have been employed with similar findings by Spector, Muraida and Marlino in their study on CBI 
authoring (1992). 

Conclusions and Implications 

Focusing students’ attention on specific content structure, in this case, on critical principles for multimedia 
authoring, has shown indications of increasing structural knowledge and retention (Beasley & Waugh, 1996) and 
keeping students on task (Shore, Erickson, Garick, Hickman, Stanley, Taylor, & Trunfio, 1992). For the demanding 
task of authoring, the Matrix that was offered to support learning, has the potential to decrease demands made on the 
mental effort that the student must expend on remembering authoring Principles and to increase effort toward 
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authoring content. As one student commented, the learners were required to, “really think about the program 
[multimedia module] in terms of how fundamentally it teaches. It gets the evaluator past the ‘bells and whistles’”. A 
balance does need to be struck between modifying the Multimedia Matrix to a point where it becomes less time 
consuming for students to use, yet still retains its connection to the original Park and Hannafin article (1993), as 
well as, continues to encourage students to revise and personalize it. 

While the matrix-based approach to instruction appears to have contributed to the development of the 
multimedia authoring skills for these learners, future studies may reveal that the flexible approach to instruction, 
particularly instruction that supports the learners’ evaluation and adaptation of the Matrix to meet their needs has a 
more significant impact on learning, more than the actual implementation of the Matrix itself (Jacobson, 1994). 
Future investigators may wish to compare the effect of other authoring frameworks, such as the Hypermedia Design 
Model (Garzotto, Mainetti, Paolini, 1995) to the Matrix when studying the implementation of authoring in courses. 
Similar to Jacobson’s Theory-to-Design framework, the matrix-based model appears, in a preliminary analysis, to 
meet some of the design criteria as a guide on discrete aspects of effective multimedia design. 
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Table 1 . Most Frequently Occurring Comments Concerning Multimedia Matrices v.l and v.2 



Response Type 


Rank 


Comment 


Positive 






Matrix v. 1 


1 


Guides evaluator 




2 


Supports consistent feedback 




2 


Focuses attention 




4 


Simplifies application of principles/easy to use 


Matrix v.2 


1 


Simplifies application of principles/easy to use 




2 


Provides effective checklist format 


Negative/Change 






Matrix v.l 


1 


Redundant 




1 


Inconsistent if used by many 




3 


Vocabulary too technical: define, simplify 




4 


Bad shape for response boxes 




4 


Add additional items 




4 


Reformat to checklist format 


Matrix v.2 


1 


Bad shape for response boxes 



Note. Rankings are based on items mentioned by more than 25% of the respondents. 
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Matrix for Guidelines for Designing Interactive Multimedia* 
Title of Multimedia Module 



Principle L Related prior knowledge is the 
single most powerful influence in mediating 
subsequent learning . 


Principle 2. New knowledge becomes 
increasingly meaningful when integrated 
with existing knowledge. 


Application 1. Is information presented to 
accommodate ranges of prior knowledge of 
learners by: 

□ layering and, abstracting, and/or providing 
various perspectives on new information? 

□ letting learners assemble their own connections 
with the concepts? 


Application 2. Are structural aids to the new 

knowledge provided by: 

□ making the structure of the overall lesson 
explicit? 

□ employing structural organizers, such as 
headings to differentiate between critical and 
subordinate information? 

□ summaries of key relationships? 

□ elaboration strategies within the program or 
generated by the learner? 


Comments: 


Comments: 


Principle 3 . Learning is influenced by the 
supplied organization of concepts to be 
learned. ~ ~ 


Principle 4. Knowledge to be learned needs 
to be organized in ways that reflect 
differences in learner familiarity with lesson 
content, the nature of the learning task and 
assumptions about the structure of 
knowledge. 


Application 3. Is information presented by audio, 
video, and text: 

□ in consistent interface conventions, such as 
windows, links, menus for related ideas? 

□ conceptually linked? 


Application 4. Do links between/among nodes go 
beyond mere random access by providing 
movement: 

□ from one concept to another? 

□ from broad concept to specific data? 

□ with bottom up or top down hierarchies? 


Comments: 


Comments: 


Principle 5. Knowledge utility improves as 
processing and understanding deepen. 


Principle 6. Knowledge is best integrated 
when unfamiliar concepts can be related to 
familiar concepts. 


Application 5. Does the student have the 
opportunity to: 

□ reflect? 

□ elaborate? 


Application 6. Are familiar visual, procedural, 
and/or verbal metaphors used for: 

□ lesson content? 

□ system interface? 


Comments: 


Comments 
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Principle 7. Learning improves as the 
number of complementary stimuli used to 
represent learning content increases . 


Principle 8 . Learning improves as the 
amount of invested mental effort increases . 


Application 7. Do sound, motion, text, and/or 
pictures present information in a way that: 

□ relates directly to the information? 

□ complements the information? 


Application 8. Are learners required to expend 
mental effort through on screen elements that: 

□ focus their attention by key information shown 
through diverse ways, such as highlighting? 

□ prompt them to predict, hypothesize, and/or 
generate new schema? 


Comments: 


Comments: 



Principle 9 . Learning improves as 
competition for similar cognitive resources 
decreases and declines as competition for the 
same resources increases . 


Principle 10 . Transfer improves when 
knowledge is situated in authentic contexts . 


Application 9. Is new and challenging 
information presented: 

□ using familiar multimedia conventions? 

□ without superfluous information? 


Application 10. Is information presented in: 
□ authentic, significant contexts? 


Comments: 


Comments: 


Principle 11 . Knowledge flexibility 
increases as the number of perspectives on a 
given topic increases and the conditional 
nature of the knowledge is understood . 


Principle 12. Knowledge of details improves 
as instructional activities are more explicit, 
while understanding improves as the 
activities are more integrative. 


Application 11. Are learners helped to: 

□ access data in a variety of ways? 

□ work with data in multiple ways? 


Application 12. Is an orientation provided to the 
new information and its organization through 
the use of: 

□ pre-questions, objectives, advance organizers, 
and/or probability statements? 

□ questions posed about the information that is 
presented? 


Comments: 


Comments: 
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Principle 13. Feedback increases the 
likelihood of learning response-relevant 
lesson content and decreases the likelihood 
of learning response-irrelevant lesson 
content. 


Principle 14. Shifts in attention improve 
the learning of related concepts. 


Application 13. Are the learners provided with 
opportunities for making: 


Application 14. Is kev data presented to gain 
attention through: 


□ ample responses about key concepts? 


□ design elements of highlighting, inverse 
display, change in color, fonts, and/or arrows? 


□ limited responses for incidentalfquestion- 
specific) information? 


□ procedural elements of a graphic overview 
(map of lesson structure) and/or prompts to 
relevant options? 

□ repetition throughout the module? 


Comments: 


Comments 


Principle 15. Learners become confused and 
disoriented when procedures are complex, 
insufficient, or inconsistent. 


Principle 16. Visual representations of 
lesson content and structure improve the 
learner* s awareness of both the conceptual 
relationships and the procedural requirements 
of a learning system . 


Application 15. Within the module are clear 
procedures given for 


Application 16. Are lesson content and structure 
interrelated by: 


□ navigating? 


□ concept maps? 


□ accessing support? 


□ graphical organizers? 


Comments: 


Comments: 



Principle 17. Individuals vary widely in 
their need for guidance. 


Principle 18. Learning systems are most 
efficient when they adapt to relevant 
individual differences. 


Application 17. Are learners helped In wavs to: 


Application 18. Does the module personalize the 
program: 


□ use the multimedia module itself, such as, how 
to access help or a glossary? 


□ at a nominal level by asking for learner's 
name, demographics, and/or preferences? 


□ use the multimedia module for learning, such 
as, feedback that is related to current status, or 
linked relationships ? 


□ at an advanced level by focusing on different 
rates of progress and providing relevant 
examples? 

□ at a conceptual level by adapting tasks to 
learners? 


Comments: 


Comments 



Principle 19 . Metacognitive demands are 
greater for loosely structured learning 
environments than for highly structured 
ones . 


Principle 20. Learning is facilitated when 
system features are functionally self-evident, 
logically organized , easily accessible , and 
readily displayed . 


Application 19. Can the learner monitor progress 
and/or learning strategies by: 


Application 20. Are on-screen elements designed 
to: 


□ accessing prompts? 


□ simplify learner's use of the module? 


□ using self-checks? 

□ creating linkages? 


□ help learners assess their progress in learning 
the concepts? 


□ asking questions? 




□ reviewing? 




Comments: 


Comments: 


Total number of checked items: 
n 


Overall comments: 



*The Matrix is based on: Park, I. & Hannafin, M. J. (1993). Empirically-based guidelines for the design 
of interactive multimedia. Educational Technology Research and Development. 43. 3, 63-85. The 20 
Principles cited in this document are quoted directly from Park and Hannafin. ETR&D is published by 
the Association for Educational Communications and Technology the copyright holder. The" 
Principles" are reprinted in the Matrix with permission of the publisher. 
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Matrix designed by Annette Sherry, Dept, of Educational Technology, College of Education, University 
of Hawaii at Manoa, 1996 



The Boulder Valley Internet Project: Teachers Mentoring Teachers 



Lorraine Sherry 
Dianna Lawyer-Brook 
RMC Research Corporation 



Overview 

The Boulder Valley Internet Project (BVIP) was a collaborative venture between the University of Colorado 
at Boulder and the Boulder Valley School District (BVSD), and was funded by the National Science Foundation. The 
premise of this five-year initiative was that the Internet and the World Wide Web (WWW) could provide a rich 
variety of tools and resources that could be used to enhance instruction and communication by students, teachers, and 
administrators. Both the project and the evaluation design evolved significantly as connectivity increased throughout 
the district, Internet tools became more user friendly, teachers and students alike acquired Internet knowledge and 
skills, and telecommunications began to gamer popular support. 

In 1991, however, when the project designers first created their expansive vision for the BVIP, little was 
known about effective integration of the Internet into a school district - especially a decentralized district like the 
BVSD that used site-based management for its decision making policies. 

The initial efforts of the project directors was to develop a plan that included the development and delivery 
of comprehensive district- wide Internet training over a three-year period beginning in July 1992. Secondary efforts 
centered around determining the impact of telecommunications on curriculum and instruction within those schools 
that were connected to the Internet and whose educators had participated in the training program. The project leaders 
also envisioned a foundation for shared curriculum implementations that could be used by educators throughout the 
district. 

The Trainer of Trainers model was used to create a core group of 26 teachers who then returned to their 
respective schools to become instructors and resource personnel for their colleagues. The objectives of the training 
program for this core group, and for later generations of participating teachers, to become proficient in the use of 
electronic mail (e-mail), to become comfortable investigating Internet resources, and to become experienced at 
exploring ways to integrate the use of the Internet into the curriculum. 

The project directors developed a full set of structured classes, unstructured open lab workshops, and 
ancillary materials to support the training program. They elicited feedback from new trainees to continually redesign 
and improve the training program as the technology evolved. Later, they created a home page on the WWW to 
access, share, and disseminate information that would be of use to educators throughout the district, and to render 
assistance to new users. 

Rather than being an isolated effort funded by a single grant, the project was part of a system-wide effort 
that both impacts and is impacted by the numerous factors in the educational system. As both the project and the 
Internet itself evolved, so did the model that was used to gather and report data from the evaluation. Teachers moved 
through various phases of growth, starting from exploration and professional development, through communication 
and generation of online learning communities, to eventual use of Internet activities in the classroom. 

Although the grant period has ended, the BVIP is continuing at a rapid pace. The hopes of the project 
organizers for integration of Internet-based activities into curriculum and instruction have not been fully met due to 
the inhibiting characteristics of significant educational reform itself, as well as the slow pace of connecting all 53 of 
the district's schools, paucity of incentives such as release time or extra pay for teachers who have taken on new 
duties as a result of their Internet expertise, and ongoing cutbacks in FTE and technical support at the building level. 
Instead of simply being viewed as a teaching tool, the use of the Internet becomes entwined within program 
development, a much more complex undertaking. This study describes the complex system that has developed from 
the initial efforts of the BVIP, and is continuing to evolve. 

Purpose and Scope of Study 

The team from RMC Research Corporation that had been tasked with evaluating the BVIP took a case study 
approach that was inductive, pragmatic, and highly concrete. They started by building a sound theoretical base, using 
the Diffusion of Innovations model of Everett Rogers (1995), the Adoption Analysis Tool of John Farquhar and Dan 
Surry (1994), and the Engaged Learning model of Jones and his colleagues (1995). The literature review and 





development of the model will be reported in a separate paper (Sherry, 1997). The scope of the evaluation began as a 
global inquiry using a survey and interviews, then shifted to a detailed examination with an embedded case study, 
various artifacts, and a set of focus groups. 

In 1993, after the completion of the first year of the project, Wolf and Black (1993) of the University of 
Colorado conducted a formative evaluation using a survey and interviews. The data suggested that the initial cohort 
of 26 peer trainers had been successfully trained. They felt they were more knowledgeable about the use of the 
Internet as a resource for teaching and learning, and they were beginning to apply their skills and knowledge in their 
own classrooms. Wolf and Black also identified five barriers that directly impacted teachers use of the Internet, and 
that have continued to affect the level of Internet usage throughout the entire duration of the project: time, access, 
training, resources, and usability. These results were in consonance with other studies of school districts that were 
building telecommunications infrastructures to connect their schools (Honey & Henriquez, 1993; Heaviside et al, 
1995; Levin, 1995). 

In 1994, RMC Research Corporation was engaged to expand upon this formative evaluation. They started 
by examining the training component to ascertain its value, both in terms of whether the training accomplished the 
short-term goals of being clear, useful, and engaging, and to determine whether or not the teachers were actually 
using their newly acquired skills in their classrooms. The team also investigated the influence of the project on the 
development of new curriculum and teaching strategies and the benefits of participation for the entire school district. 
Toward the end of 1996, they evaluated the BVIP as a whole to determine whether it may be easily replicable or 
transportable to other districts, and how it may be improved. 

An Integrated Technology Adoption and Diffusion Model emerged from the BVIP evaluation that comprises 
not only the technological and organizational factors found by Wolf and Black, but also individual and instructional 
factors that were revealed in the research of Rogers, Farquhar and Surry, and Jones et al. It is somewhat similar to the 
model recently developed by Lewis and Romiszowski (1996). RMC Research Corporation’s integrated model melds 
three approaches by which one can view the barriers to technological innovations and the factors that either enhance 
or inhibit the change process by the adopting educational organization. One may view technology adoption from the 
perspective of access, cost, type and age of available computers and hardware, and the physical aspects of the school 
network, reliability, and user interface. One may consider the viewpoint of the user, encompassing both user 
characteristics and the users perceptions of the innovation. Or, one may focus on the complex needs of the 
educational institution itself - a school or district situated within a community, which, in turn, is situated within a 
set of policy-making bodies and the local culture as a whole. 

After compiling and reviewing the research on the four strands of variables that impact the adoption and 
diffusion of a new information technology innovation, and after collecting the initial data via surveys and interviews, 
the team created a model that targeted the most important variables and applied it to the BVIP. As the data collection 
proceeded, and as the information was coded and sorted, the model was expanded. Figure 1 presents a summary of all 
of the factors that influenced the adoption and diffusion of the BVIP throughout the BVSD. 

Pilot studies of a technology implementation plan based on the Integrated Technology Adoption and 
Diffusion Model that emerged from the BVIP evaluation are planned for the spring of 1997. 



Technological Factors 

Access, reliability, usability, network response time 

Individual Factors 

User characteristics: motivation, need for control, expertise, comfort level, reasons and patterns of use, gender, 
special needs 

Perceived attributes of the innovation: relative advantage, observable benefits, complexity, compatibility with 
needs and wants, capacity for experimentation on a limited basis 

Organizational Factors 

Physical environment: classroom connectivity, network capacity, availability of equipment and supplies, 
scheduling of labs and computers 

Support environment: administrative vision and support, district policies, communication, problem-solving 
mechanisms, training, availability of support, maintenance, incentives, cooperation with other funding sources 

Instructional Factors 

Curriculum: change in content, volume of content, curriculum enhancement, planning and preparation, standards, 
use and sharing of lesson plans, evaluation and categorization of Internet activities 





Engaged learning: teacher and student roles, collaboration, learning context, generative learning, authentic tasks, 

multidisciplinary studies, authentic assessment 

Figure 1. Integrated Technology Adoption and Diffusion Model 

Study Methods and Data Sources 

Because the purpose of the study was to build an expanded theory base as well as to provide formative and 
summative evaluation information to the funding agency, the evaluation was conducted as a case study. According to 
Yin (1994), case studies are generalizable to theoretical propositions, and not to populations or universes. The 
applicability of the BVIP model to other school districts will depend in large part upon the districts administrative 
vision and support, and upon the structure of the decision making process, whether site-based or centralized. 

Setting 

The BVSD is a large, dispersed, partially mountainous district comprising 53 elementary, middle, and high 
schools, and is situated to the northwest of Denver. The City of Boulder is home to the University of Colorado. 

An embedded case study was conducted at Nederland Elementary School, an isolated, rural K-5 school 
situated in a small mountain town within the BVSD. It is a high-end user school that made good use of the Internet 
to expand the learning environment beyond the walls of the classrooms and beyond the small community of which it 
is a part. It is also one of the few schools in the district with schoolwide connectivity. 

The evaluation team focused on this school because it provided an opportunity to gather data from teachers 
and technology resource people who were actually using the Internet as an integral part of teaching and learning. 
Though connectivity was available to students in all grades, the primary use was by the fourth and fifth graders. 
Hence, the data do not represent a random sampling across all grades; they reflect the ideas and activities of the 
teachers and students who were high-end users. 

Evaluation Design 

After reviewing the relevant research literature and discussing the project with the BVIP project director, five 
research questions were formulated that underlie the investigations carried out in this case study. Figure 2 
summarizes the five research questions and the data collection activities. 
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What was the impact of 
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The district? 
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X 


X 


What are the possible 
future uses of this model? 
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Figure 2. Data Collection Strategy Matrix 

Because the BVSD was a complex system, multiple measures were used to produce converging lines of 
inquiry. This approach involved both qualitative and quantitative methods, as seen in Figure 2. Throughout the 
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study, strict confidentiality was observed. Quotations gathered from transcripts were coded and then checked by a 
second team member for inter- rater reliability. 

Initial information on the effectiveness of the training component was gathered from a district-wide e-mail 
survey. The quantitative data were supplemented and enriched by the follow-up interview of teachers who were using 
the Internet in their classrooms. The documentation analysis involved examining the BVSD system logs, previous 
project documents and evaluations, newspaper articles, white papers, and other artifacts. More detailed data were 
obtained through the embedded case study and the curriculum focus group. The Internet activity classification project 
generated by the work group will be used to assist BVSD teachers in classifying instructional activities and units 
that use the Internet effectively. 

Results 

The Integrated Technology Adoption and Diffusion Model proved effective for collecting, organizing, and 
reporting the findings from all phases of the evaluation. The information that was gathered was then used to answer 
the five research questions. If this model is to be used for creating technology plans for other districts, some 
restructuring of the instructional and organizational factors may be required. 

Research Question 1. How effective was the training component of the project? 

Overall, the BVIP was successful in meeting the goals of its training component, namely, to develop and 
deliver comprehensive district-wide teacher training in the use of the Internet and the BVSD network for educational 
purposes. The project leaders are still studying the impact of the project on curriculum and instruction. The 
establishment of a foundation for shared curriculum applications is yet to be achieved, but project participants have 
made good progress in this direction with the creation of the BVSD home page, several school home pages, and 
various curriculum implementation strategies that they have shared with their colleagues. 

The Trainer of Trainers model was an effective strategy in disseminating Internet training throughout the 
district. It has also been used successfully in similar projects. (See Main, 1996.) Starting with a cohort of 26 
teachers, the project team significantly expanded the level of telecommunications expertise and usage throughout the 
district. As of January 1996, there were 435 regular e-mail users on the BVSD network. The training programs focus 
on peer training of classroom teachers, and the flexibility of the training in keeping up with the rapid changes in 
communications technology, were particularly effective strategies. 

The barriers encountered by participants were primarily organizational in nature, such as insufficient staff 
development time, delays between training and access at the school level, and cuts in district funding in a variety of 
areas. More importantly, since the training was aimed at typical classroom teachers rather than administrators and 
policy makers, expertise was diffused horizontally at the grassroots level, rather than percolating upwards toward all 
levels of the educational system. 

Research Question 2. How did the project specifically affect the participants' use of the 
technology? 

The most significant influence on the participants use of technology was the increase in communication 
with colleagues, experts, and friends within the building, within the district, or outside the district. Originally, the 
Internet was envisioned as a tool for research; over the course of the project, the communication aspect achieved 
equal or even greater importance. Using e-mail for communication and accessing the WWW for both teacher- and 
student-initiated research were the most popular uses of the Internet. Teachers who became proficient in Internet use 
accessed many educational resources that supported their professional growth. Students, too, became proficient at 
using e-mail to contact their friends, students at other schools, or experts in subject areas that they were researching. 

Teachers recognized that telecommunications is a potentially useful tool for all students and staff. Creative 
solutions by several innovative teachers who used the WWW for special populations showed great potential to avoid 
inequities. One interesting example discussed in the focus group concerned a young student with attention deficit 
disorder who became fascinated with Sea World's online resources on manatees. As a result of working with these 
resources in class, he became more calm, settled, and concentrated as the semester progressed. 

Barriers to the use of technology included an increased work load for some new trainees, lack of observable 
benefits for the additional training involved, and declining morale because of the district funding cuts and the 
reassignment of in-building technology resource personnel to the classroom. 



O 

ERJC 



396 



384 



Research Question 3. What was the impact of the project on curriculum and instruction? 

The impact of the project on teaching and learning was strongest when the classrooms had direct access to 
the Internet and when the teachers instructional style matched their uses of the Internet. The focus of the curriculum 
shifted from developing technology skills to enriching instructional content. 

Teachers' need for control influenced the ways in which they used the Internet for instructional purposes. 
Loss of control was due in part to the open framework of the Internet, and in part to the rapid gain in expertise by 
the students, which sometimes resulted in a novice/expert role reversal between students and teachers. Some teachers 
were also concerned about if and what subject content would be replaced by Internet-based activities. 

Teachers who saw themselves as facilitators were mote able to adapt to their role as guide, coach, co- 
explorer, and co-learner with their students than those with a more traditional pedagogy. Teachers who were able to 
tap the expertise of their students and use them as assistants in class projects found that their workload was decreased 
by shifting some of the responsibility of finding suitable resources to their students and using their students as an 
informal support network. Those who regularly used the Internet for class-related purposes had a variety of strategies 
that they used to plan and prepare for their classes, including finding curriculum-related resources in databases on the 
WWW, participating in mailing lists with colleagues and experts who shared their interests, and sharing information 
and activities with other teachers and associates throughout the district. 

The volume of content on the Internet continues to pose a problem for teachers who need quick and ready 
access to relevant instructional materials. Teachers who participated in the summer work group designed a 
classification scheme for organizing Internet resources and relating their content to the curriculum. This classification 
could be used by the BVIP to develop a foundation for curriculum applications. This will become increasingly 
important with the current shift by the school board to the adoption of district-wide standards and assessments. 

Research Question 4. What was the impact on the school as a whole? the district as a whole? 

Regarding the impact of the project on a whole school, the embedded case study of a rural mountain 
elementary school revealed a wide range of expertise among the staff. Some teachers had a high level of expertise, 
whereas the support staff hardly used the network at all. Students and teachers with home access used the Internet 
more at school than those who lacked home access. 

Age had little effect on expertise. Some young students were quite adept at Net searching, communication, 
designing and building home pages, and exploring the Internet for their class projects, whereas some teachers felt 
intimidated by the Internet and hardly used it at all. 

Gender impacted both the amount and type of use, with boys using the Internet more for exploring and girls 
using it more for communication. Both boys and girls, however, used the WWW regularly for research projects. 
Some generated their own HTML code for building Web pages and were able to demonstrate their skills to the 
evaluation team. In particular, a multi-age science class comprised of students from Nederland's elementary and 
middle school collaborated with Jason Project researchers to create a home page for their class and several products 
for the Denver Natural History Museum, all focusing on oceanography. 

Growth in the use of the Internet throughout the district as a whole was promoted by the students 
enthusiasm. As one teacher remarked, the students are pushing the technology. Over the course of this study, 
however, growth was hampered by the limited connectivity. January 1997 was the target date for full district- wide 
connectivity. However, classroom connections will be deployed slowly as individual schools raise funds to build 
their own local area networks. 

Two diffusion models were originally proposed but were not implemented One was a vertical model in 
which middle school students were to mentor elementary school students, and high school students were to mentor 
middle school students. The vertical model was not used because the middle schools had Internet connections from 
the beginning of the project, but the elementary and high schools did not get their Internet connections until much 
later. The other, a horizontal model connecting three middle schools, was not pursued because, at the time, it was 
decided that this innovation entailed changes in curriculum. With the incorporation of standards and the ensuing 
curriculum revisions that were taking place, this was simply not possible. 

Diffusion of telecommunications throughout the district was successful at the grassroots level because of 
the empathy between the cadre of peer trainers and their teacher-trainees. Besides the lack of incentives in the way of 
release time and a lightened workload for teachers who provide inservice training or support for their colleagues, 
another major barrier to the effective use of the Internet throughout the district is the lack of an online, centralized 
bank of activities and resources, grouped by grade level and content, that are directly related to the districts current and 
planned curriculum. 
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Research Question 5. What are the possible future uses of this model? 

To answer this question we must focus on the actual successes of the BVIP. First, it was a grassroots effort 
that successfully diffused horizontally among those classroom teachers who participated in the training sessions. 
Second, the Trainer of Trainers model worked very well because it capitalized on the empathy between change agents 
and clients. Third, the project itself was successful at first because it had initial buy-in by the superintendent and the 
school board - the main policy making bodies. It was only after the superintendent and school board were replaced by 
more conservative individuals that the district support began to falter. And fourth, the project worked successfully 
with other grants and sources of support such as the Annenberg/CPB Math and Science Project, the district bond 
funding, and the University of Colorado. 

The BVIP proved to be a dynamic, evolving program housed within a decentralized educational organization. 
The reality of the BVIP was quite different ffom the theory base of the Rogers Diffusion of Innovations model. 
Change has been slow due to the gradual process of connecting all 53 schools within the district, but it continues 
due to the solid base of expertise that has been built throughout the projects five year effort. Future plans for the 
project include access at all schools and greater attention to curriculum content and instructional strategies. 

Equity of access issues, too, were important. The levels of Internet use, comfort, and expertise acquired by 
those who had home access were generally greater than among those who did not have home computers. Equity is 
not limited to the BVIP alone, but is an issue that must be dealt with by any district that intends to use Internet- 
based activities as part of the curriculum. 

Other districts that are considering replicating the BVIP efforts should consider their own style of 
management and decision making processes when designing approaches to the adoption and diffusion of Internet- 
based classroom activities. 

Conclusions and Recommendations for Further Research 

Besides providing ongoing information to the district and the project leaders, as well as feedback at the end 
of the project concerning its degree of success in meeting its objectives, the evaluation produced an expanded theory 
base and an Integrated Technology Adoption and Diffusion Model that can be used for other districts that plan to 
adopt the use of the Internet in their schools. For this model to be effective as a planning tool, rather than strictly as 
an evaluation tool, however, some changes must be made in the model. Specifically, it is important to concentrate 
on the interaction of the teachers pedagogical styles with learning objectives and the value that can be added to the 
instructional process through the use of Internet-based activities, and design a staff development plan that addresses 
the needs of the teachers in the target district. 

Moreover, it would be advisable for any planned staff development program to address administrators and 
policy makers as well as classroom teachers. Since the use of the Internet becomes an integral part of program 
improvement, it is important for those who are in charge of setting and carrying out district policies - including the 
school board, the superintendent, parent-teacher organizations, school principals, and school administrators - to be 
involved in both supporting and participating in the training activities, so that they can create, sustain, and 
communicate a clear administrative vision throughout the entire educational system. 

Student success stories have been instrumental in promoting the use of the Internet within those schools 
that are already connected, via student-produced home pages. It is important to have policies in place that allow 
students to use the Internet during their ffee time, before or after school, especially if they do not have access at 
home. 

It is also important to have policies that encourage schools and classes to place their home pages on a 
centralized location, that offer support to new users, that steer them to the right people and resources when they 
encounter problems with the network, and that provide ongoing maintenance for the system, not only at the district 
level, but at the local building level as well. In schools where principals and technology resource people provided 
vision and support for technology, the use of technology flourished. 

Although teachers are aware of the many possibilities that the Internet offers for enhancing teaching and 
learning in the classroom, many need further guidance in strategies for integrating telecommunications into their 
curriculum and instruction. There are three important areas that need further exploration: 

How can or will teachers use the new technologies? 

How can or will the new technologies fit in with teachers current teaching styles? 

What value can technology add to the teaching and learning process? 
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Future research efforts will be devoted to answering these questions and reformulating the Integrated 
Technology Adoption and Diffusion Model into a form suitable for a district technology implementation plan. 
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Abstract 

Research on the motivational aspects of multimedia games may provide ways to design more engaging user 
information systems which increase users' esploratory and information-seeking behaviors. Two small-scale 
exploratory studies examined the effects of introducing information on the intrinsic motivation of users of a CD- 
ROM game. Results of Study I showed a negative relationship between age and both trait and state curiosity , a 
negative relationship between tolerance for ambiguity and state curiosity. Study I showed a significant decrease in 
state curiosity after subjects received informational clues while Study II found that subjects who received 
informational help sheets had significantly greater curiosity. 

Introduction 

Researchers of information systems often focus on the user performance (e.g. learning) during a human- 
computer interaction and ignore motivational factors that affect that experience. Research on computer learning 
games has revealed that they are most powerful for generating and enhancing interest and motivation (e.g. 
Chartier,1973; Cherryholmes, 1966; Klein and Freitag, 1991; Malone, 1981). 

Investigations which look at various motivational variables of users while using computer games may 
provide insight into (1) designing more engaging user interfaces, (2) increasing users 1 intrinsic motivation and 
feelings of competence, and (3) stimulating better and greater use of other types of information systems (Trevino & 
Webster, 1992; Webster, Trevino, & Ryan, 1993). 

This research explored the relationships of several motivational variables and the introduction of 
information during an ambiguous but highly engaging task. Two pilot studies explored curiosity and flow as 
indicators of intrinsic motivation for users of the multimedia CD-ROM game Myst (Broderbund Software, Inc. and 
Cyan, Inc., 1994). 

Intrinsic Motivation 

Deci (1975) describes intrinsic motivation as an innate need, founded on competence and self-determination, 
to explore one’s environment. Intrinsically motivating activities are those that stimulate play, exploration and the 
development of cognitive structures through challenge and incongruity (Deci, 1975). Malone (1981) describes an 
intrinsically motivating activity as one in which people engage in it for its own sake, rather than for some external 
reward. Intrinsic motivation may be demonstrated by a demonstrated by a range of motivational factors. This research 
focused on two seemingly-related factors— curiosity and flow — of subjects engaged in an ambiguous task. 

Curiosity 

Berlyne (1960) describes "curiosity M as exploratory behavior in search of an optimal state of arousal and 
epistemic curiosity as an internalized search to resolve some conceptual conflict and achieve knowledge. Day (1982) 
differentiates between trait curiosity, a general propensity for being curious, and state curiosity, a demonstration of 
curious behavior toward a specific thing, task, event, or situation. 

B oy kin and Harackiewicz ( 1981) found a positive correlation between curiosity and uncertainty. Ingwersen 
(1992) defines uncertainty as "a state of doubt in which the individual's own state of knowledge, work space and 
cognition cannot fill the problem space by thinking, causing interaction with the world around it to obtain 
supplementary information" (p. 131). As a person encounters large amounts of unique information, uncertainty and 
anxiety result. Kuhlthau (1993) describes a "tolerance for uniqueness" at the beginning of the information search 
process until new constructs are built and redundancy occurs. This is consistent with Berlyne’s (1960) description of 
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“specific curiosity” — curiosity provoked by a lack of information about a complex, novel, and incongruous stimulus 
and leading to exploratory behavior designed to gather information in order to reduce uncertainty. 

If curiosity and uncertainty are related, then a curious person would likely have a high tolerance for 
ambiguity. One who is intolerant of ambiguity perceives ambiguous materials or uncertain situations as unpleasant 
or threatening (Budner, 1962); while one who is tolerant of ambiguity may find the same materials or situations 
comfortable or even desirable and may actually seek out ambiguity and ambiguous situations that lack adequate 
structure of sufficient cues (MacDonald, 1970). Computer games that present an ambiguous but challenging task, 
therefore, may stimulate curiosity. 

Malone (1981) considers the degree to which curiosity can be aroused and then satisfied as "one of the most 
important features of intrinsically motivating environments" (p. 337). He identified three additional factors 
(challenge, control, fantasy) that influence initial and continuing engagement in a task. These factors (except fantasy) 
are similar to the dimensions of flow, as described below. 

Flow 

Flow has been defined as "the process of optimal experience" (Csikszentmihalyi, 1975; Csikszentmihalyi & 
LeFevre, 1989) in which there is a satisfying, even exhilarating, feeling of creative accomplishment and heightened 
functioning. A flow state is one in which a person (1) suspends time and space while fully immersed in a 
challenging activity, (2) focuses attention on a limited stimulus field which provides clear and unambiguous feedback 
on actions taken, (3) perceives a sense of control, and (4) finds the experience, itself, rewarding (Csikszentmihalyi, 
1975; 1990). Bialeschki & Henderson (1992) describe flow as an intersection of high skill level and high challenge. 
When the challenge is too high and the person's skill too low, anxiety is experienced; when the challenge is too low 
and the skill too high the person experiences boredom. A combination of low challenge and skill results in apathy. 

People of all ages can experience flow in both work and leisure activities (Csikszentihmahlyi & LeFevre, 
1989). Reiber (1996) asserts that "flow theory provides an important framework for an adult’s motivation for 
learning" (p. 48). Research has shown that people will seek to replicate or find flow in the activities and events of 
their lives. Webster, Trevino and Ryan (1993) studied the playfulness of employees, using various application 
software in order to develop a measure of flow. They identified three dimensions of flow — control, focused attention, 
and enjoyment (combining curiosity and intrinsic interest). In addition, Csikszentihmahlyi (1990) found that 
providing clear requirements and immediate feedback also facilitated the flow experience. 

Trevino and Webster (1992) recommend designing computer-mediated communication technologies that 
provide more user control, focus the user's attention, and incite curiosity and interest in order to increase positive 
attitudes and more positive changes in communication-related outcomes. Researchers (e.g. Carroll and Thomas, 
1988; Davis, 1989) have articulated need for research on playfulness in various types of information systems. 

Amone and Small (1994) contend that there may be a parallel between Csikszentihmahlyi's concept of flow 
as optimal challenge and Berlyne's concept of curiosity as optimal level of arousal. However, the distinction between 
these two concepts is that curiosity requires some kind of conceptual conflict to be resolved while flow does not 
(Rotto 1994). Malone (1991) suggests that curiosity is aroused during the flow state. In computer-based 
environments, this can be accomplished through varied, novel and surprising stimuli such as color and sound and 
through providing options (e.g. menus) that encourage exploration (Trevino and Webster, 1992). 

All of the factors used to describe a curious person or a person experiencing flow might also describe many 
users of computer games. "Simulations and games seem to embody many of the aspects of stimuli which theorists 
have found to increase motivation and maintain it at an optimal level" (Spitzer, 1976). All one needs to do is 
observe a game player in a video arcade or watch a Nintendo user to recognize intrinsically motivated behaviors 
(Rezabek, 1994), especially curiosity and flow. Many of these games' design elements (e.g. challenging problem to 
solve, goals with uncertain outcomes, immediate feedback, engaging task, interactive format) combined with user 
factors (e.g. intense concentration, a sense of control, the need to resolve cognitive conflict or incongruity) facilitate 
flow in this environment (Hoffman and Novak, 1995). 

Cruickshank and Telfer (1980) list some specific advantages of simulations/ games as providing 
opportunities for (1) active participation, (2) problem-solving experiences, and (3) a responsive, engaging, safe, and 
enjoyable environment. Research that identifies some of the motivational elements of computer games and potential 
interventions that influence user motivation may be helpful for understanding how to design and organize 
information systems that stimulate the intense level of concentration, high curiosity level, perception of control, and 
enjoyment associated with the flow experience or that identify software that has "flow potential" ( Bialeschki & 
Henderson, 1992, Mandler, 1984). These two pilot studies explored the relationships among motivation variables 
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and the impact of providing information to reduce uncertainty on subjects playing the highly-rated multimedia CD- 
ROM game, Myst. 

Myst requires the individual user to explore a fantasy island and, through a series of visual puzzles, attempt 
to solve the island's mysteries. Named one of the best CD-ROMs of 1994 (Consumer Reports, 1995; Fortune, 
1994; MacUser, 1994; New Media, 1994), Myst was chosen for this research because of its high motivation 
potential — beautiful and sophisticated 3-D graphics, mystical music and sound effects, and a challenging problem- 
solving format. Furthermore, it was especially appropriate for this research because of its high degree of ambiguity 
and its fantasy environment as players seek to identify and determine the dimensions of its problem situation. Myst 
appears to stimulate both types of curiosity described by Malone (1981) — sensory curiosity that is stimulated by an 
enriched learning environment (e.g. animation, music) and cognitive curiosity that is stimulated by an incomplete, 
inconsistent, unparsimonious environment. 

Although highly engaging, Myst provides an ambiguous task for users through no verbal clues, uncertain 
goals, limited direction. It is what Bruner (1986) calls "an interpretive task," occurring when a person first 
encounters new information and requiring the person to recognize patterns in order to make inferences and 
predictions. Kuhlthau (1993) describes the information seeking process as one "in which users progress from 
uncertainty to understanding" (p. 345). 

This paper reports the results of two small-scale, exploratory studies. Because the flow experience depends 
so heavily on focused attention on and control of a task, this research used a highly engaging task that provided 
(initially) no rules or goals to explore studies explored the effects of introducing an information intervention after 
subjects were fully engaged in the task on a range of motivational variables. The first study investigated the 
relationship of a users' tolerance for ambiguity, curiosity, and user beliefs and feelings, while second study focused 
on flow dimensions (attention focus, control, intrinsic interest, and curiosity). 

Study I 

The objective of Study I was to investigate the relationship of tolerance for ambiguity, trait and state 
curiosity and several feelings and beliefs of users while engaged in an ambiguous task (computer game). Research 
questions were: 

(1) What is the relationship of motivation variables (tolerance for ambiguity, trait curiosity, state curiosity, user 
beliefs, user feelings) to user characteristics (age, gender)? 

(2) What are the relationships among motivation variables? 

(3) Does introduction of informational clues midway through a highly engaging, ambiguous task affect user 
motivation? 

Subjects were 31 volunteer undergraduate students at a mid-size, northeastern university. There were 22 
males and nine females, ranging in age from 19 to 32 with a mean age of 21. None of the subjects had ever played 
Myst. Instruments used were: 

• Tolerance for Ambiguity Scale (AT-20) (MacDonald, 1970). This scale required marking twenty 
statements as either true or false. (Examples: "There's a right way and a wrong way to do almost everything." 
"Sometimes I rather enjoy going against the rules and doing things I'm not supposed to do.") The AT-20 Scale's 
split-half reliability was .86 and evidence for construct validity supported the hypothesis that high tolerance people 
tend to succeed in ambiguous tasks (MacDonald, 1970). 

• Melbourne Curiosity Inventory (MCI). The two forms of this instrument were utilized in this study. The 
Trait Form (Naylor, 1981) measures a general capacity or propensity to experience curiosity (Loewenstein, 1994). It 
consists of twenty statements required ranking from 1 (almost never) to 4 (almost always). (Examples: "I think 
learning 'about things' is interesting and exciting." "I like to enquire about things I don't understand."). Naylor 
(1981) found this measure had high test-retest validity. The State Form (Naylor, 1981) measures curiosity toward a 
specific situation or event. It consists of twenty statements required ranking from 1 (not at all) to 4 (very much so). 
(Examples: "I feel curious about what is happening." "My interest has been captured."). Both forms are reported to 
have high reliability (e.g. Olson and Camp, 1984). 

• User Dimension Scales (Small. 1995). This instrument contained two sub-scales. The User Beliefs sub- 
scale consisted of a nine-point Likert-type scale containing nineteen bipolar terms, derived from validated cognitive 
measures of flow developed by Trevino and Webster (1992) and Ghani, Supnick and Rooney (1991). These terms 
measured users' cognitive perceptions about the game (e.g. difficult-easy, consistent-inconsistent, familiar- 
unfamiliar) along a 7-point scale. The second sub-scale. User Feelings required subjects to rate their motivation on a 



O 

ERIC 



403 



390 



five-point scale ranging from 1 (not true) to 5 (very true). Ten items were considered positive terms (e.g. in control, 
excited, confident) and seven were negative (frustrated, confused, bored). Items were derived from the literature. 

Procedures 

Several days prior to the treatment, subjects were administered the Tolerance for Ambiguity Scale and the 
MCI-Trait Form. The treatment required subjects to spend up to one hour playing the multimedia CD-ROM game 
Myst. The treatment was conducted in a computer research lab environment. As subjects began the session, he/she 
was told that the directions for the game had been misplaced and he/she would have to try to figure out the game 
without any information. 

After 30 minutes, subjects were told to stop playing the game and complete MCI — State Form and the User 
Dimension Scales. Before continuing the game, subjects were told that the game directions had been found and they 
were allowed to use them during the last half of the treatment. The directions, supplied by the game makers, 
consisted of three brief hints. At the end of the hour, subjects were again administered the MCI-State Form and the 
User Dimension Scales. 

Results: Study I 

Data were analyzed according to each of the research questions. 

(1) What is the relationship of motivation variables to user characteristics (age, gender)? 

Results reported in Table 1 show that in this study, age had a negative relationship (p<.05) with both (post) 
trait and state curiosity; i.e. the older the subject, the less curious he/she was. Age had a positive relationship with 
(post) user feelings; i.e. user's feelings toward the task increased with age. 
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Table I. Correlation Coefficients for Motivation Variables by Age. 



The only gender difference occurred on the (post) state curiosity dimension where females had significantly 
higher scores (p<.05) than males (see Table 2). All motivation scores showed a slight decrease from mid to post 
measures for both males and females. 
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(2) What are the relationships among motivation variables? 

Table 3 reveals a negative relationship (p<.05) between tolerance for ambiguity and (both mid and post) 
state curiosity. There was also a negative relationship (p<.05) between tolerance for ambiguity and both (mid) user 
dimensions (p<.05) and post) user dimensions (pc.Ol). Thus it appears that in this study, the higher the level of 
tolerance for ambiguity, the lower the level of state curiosity and post user dimensions . Additionally, mid and post 
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state curiosity were significantly related (jk.OOI) and so were mid and post user beliefs (p<.05), while the correlation 
between mid and post user feelings approached significance (p= 09). State curiosity (mid) was also significantly 
correlated with both user beliefs (mid) and user feelings (mid) (p<.05). State curiosity (post) was related to both user 
beliefs (post) (p<.01) and user feelings (post) (p<.05). User beliefs (mid) were significantly correlated with user 
feelings (mid) (pc.001) and user beliefs (post) were related to user feelings (post) (p<.001). 
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Table 3. Correlation Coefficients for Motivation Variables. 



(3) Does introduction of informational clues midway through a highly engaging, ambiguous task affect user 
motivation? 

(Post) state curiosity scores showed a significant decrease from (mid) state curiosity scores after receiving an 
information help sheet (p<.05) (see Table 4). Differences in user dimensions were not significantly different; 
however, a mid-to-post decrease in scores was noted, with the decrease in user feelings approaching significance 

(p=.06). 
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Table 4. Correlated T -Test for Mid and Post Motivation Variables . 



Discussion 

This study sought to determine relationships among various motivation variables and user characteristics 
and the effects of receiving information while engaged in an ambiguous task. Subjects playing the CD-ROM game, 
Myst were provided with an information help sheet at the mid-point in the treatment. One result, a negative 
correlation between tolerance for ambiguity and both mid and post state curiosity is contrary to findings by Boykin 





405 



and Harackiewicz (1981) and appears to contradict the very nature of curiosity; i.e. that it is stimulated by 
incongruity and uncertainty in the environment. 

The negative correlation between age and both trait and state curiosity is consistent with past research. Day 
(1982) describes a decline of curiosity with age when there is more emphasis on specificity and extrinsic rewards 
during formal education. The resulting significantly higher state curiosity for females than males after playing the 
game was an interesting finding, particularly given the small number of female subjects in this study. 

Both state curiosity measures were highly related to each of the user dimensions. A person’s curiosity level 
may affect both their positive beliefs about their abilities for achieving the game's challenge and their desire to do so. 
In addition, the informational clues appear to have had some affect on final curiosity levels, as state curiosity scores 
decreased significantly after receiving them. Yovits and Foulk (1985) found that information may increase rather 
than decrease uncertainty in some situations. 

Study II 

The purpose of Study II was to explore how the introduction of information affects subjects were engaged in 
an ambiguous task affected their motivation. In this study, motivation variables comprised the dimensions of flow — 
attention focus, control, intrinsic interest, curiosity. Research questions were: 

(1) Is there a relationship between motivation variables (attention focus, control, intrinsic interest, curiosity, total 
flow) and specific user characteristics (age, gender)? 

(2) Does the introduction of an informational help sheet midway through a highly engaging but ambiguous task 
affect subjects' motivation? 

(3) Are there differences between treatment groups on mid and post motivation scores? 

(4) Is there a relationship between level of excitement, challenge, and frustration and each of the motivation 
variables? 

(5) Does level of motivation affect the desire to engage in the same task again? 

Subjects were 29 volunteer undergraduate students (22 males; 7 females) at a mid-sized, northeastern 
university. None had ever played Myst. Subjects were randomly assigned to one of two treatment groups — one that 
received an informational help sheet midway through the session and one that did not. 

Subjects were asked to spend two hours in a computer research lab playing the multimedia CD-ROM game 
Myst. As they began the session, they were told that the directions for the game had been misplaced and they would 
have to try to solve the mystery of the game without them. 

After 45 minutes, subjects were told to stop playing the game and complete a nine-point Likert-type scale 
that required them to rate their feelings at that point in time on 23 opposite terms (e.g. focused-unfocused; free- 
constrained; excited-bored; curious-disinterested). Terms included the 13 items from Csiksentmihalyi and Larson's 
(1987) Experience-Sampling Form and ten additional items derived from the literature (e.g. Ghani, Supnick and 
Rooney, (1991). Personal data (age, gender) were also collected. 

Before resuming the game, one treatment group was given a one-page paper containing specially-developed 
information for solving some of the game's most difficult elements, while the other treatment group was given no 
additional information. After 45 minutes, subjects were again administered the same Likert-type scale and were also 
asked to rate their level of excitement, challenge, and frustration with playing this game. Subjects receiving the 
information were also asked four questions about the helpfulness of the information provided. 

Results: Study II 

Data were analyzed according to each of the research questions. 

(1) Is there a relationship between motivation variables (attention focus, control, intrinsic interest, curiosity, total 
flow) and specific user characteristics (age, gender)? 
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In (both mid and post) treatment scores, there appear to be no significant relationships between any of the 
motivation variables and age. However, as indicated in Table 5, males scored significantly higher on (mid) total flow 
measures than females (p<.05) and approached significance on (mid) intrinsic interest (p=.08). 

(2) Does the introduction of an informational help sheet midway through a highly engaging but ambiguous task 
affect subjects' motivation? 

It is worthwhile to compare post-treatment scores only when comparing means between subjects who 
received information and those who did not, since the information was provided immediately following 
administration of the mid-treatment instruments. Results, reported in Table 6, indicate that subjects who received 
the informational help sheet had higher curiosity (p<.05) and feelings of control (approaching significance at p= .08) 
and total flow (approaching significance at p=.09). A subsequent t-test comparing mid to post-scores on all 
motivation variables showed no significant differences. 
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Table 6. Group/Independent T-Test of Flow Variables for Both Treatment Groups . 



(3) Are there differences between treatment groups on mid and post motivation scores? 

Results indicate no significant differences from mid to post treatment scores for subjects in either treatment 
group on all motivation variables (see Table 7). Thus, it can be concluded that in this study, the introduction of 
information at mid-point in the treatment did not affect motivation level. 
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Table 7. Correlated T-Test for Mid and Post Scores on Flow Variables for Both Treatment Groups. 



(4) Is there a relationship between level of excitement, challenge, and frustration and each of the motivation 
variables? 

Each of the motivation variables was highly positively correlated with level of excitement (see Table 8). 
Challenge, however, although significantly correlated with curiosity (p<.05), had no relationship to the other 
motivation variables. Frustration, on the other hand, was significantly negatively correlated 
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Table 8. Correlation Coefficients for Excitement, Challenge, and Frustration on Flow Variables for All Subjects. 
with all motivation variables. Thus, the more frustrated subjects felt afterplaying the game, the lower their scores on 
all motivation variables. Additional analyses were conducted to determine whether there were differences between 
treatment groups on excitement, challenge, and frustration levels and no significant differences were found. 

(5) Does level of motivation affect the desire to engage in the same task again? 

Subjects were asked "Would you play this game again on your own?" An independent t-test of motivation 
variables and desire to play the game again revealed significantly higher intrinsic interest (p<.001), curiosity and 
total flow (p<.05) for subjects who said they wanted to play the game again (see Table 9). Differences in attention 
focus scores approached significance (p= 07). 
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Table 9 Independent T-Test of Several Variables on the Desire to Play Again 



Subjects in the information treatment group were asked three questions about the relative usefulness of the 
informational help sheet for solving the game’s problem, for moving through the game, and for understanding the 
game’s structure and one question as to whether they liked receiving the information. Of the 14 subjects receiving 
information, 1 1 (78%) indicated they appreciated receiving them, stating they gave them direction, options, and 
alternatives to try. "The hints only provide a little direction instead of the user being lost totally" one subject wrote. 
Another subject indicated that after receiving the information, he advanced a little more but not enough to be 
satisfied. Another wrote, "(I)t would take long to figure it out on my own, however, I was still having problems 
after receiving the (information).” Of the three subjects reponding negatively about receiving information, two said 
they had already figured it out by themselves and one said it didn’t provide enough information. Thus, in general, it 
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appears that the information did not afford much help for solving the problem, and therefore, did not affect subjects' 
motivation. 

Discussion 

This study indicates that the introduction of information to subjects at mid-point in an engaging but 
ambiguous task had little effect on their overall flow. In an evaluation of six studies on the effectiveness of 
instructional simulation games, Cherryholmes (1966) posited that lack of effects may have been due to presenting 
subjects with the simulation rules and that the effectiveness of simulation games may come from students 
discovering those rules. Reiser and Gerlach (1977) further state that it is the structure, rather than the content, of 
simulation games and its mastery that causes high student interest. Subjects in this study appear to have found the 
information helpful in understanding the game's structure. 

Conclusions and Recommendations 

There may be other explanations for the lack of effects of receiving relevant information while playing the 
game Myst . One is the small number of subjects in these studies, which made it difficult to identify real differences 
among groups, hence the use of t-tests rather than analysis of variance. Another is the challenge of playing this 
particular game may have been too high so that users' skills did not match the level of challenge (a requirement of 
flow), possibly causing high levels of frustration as, indicated in the results, and the quality of amount information 
may not have been sufficient to lower these levels. 

These two exploratory pilot studies attempted to determine the effects of introducing an informational 
intervention on undergraduate students' motivation while playing a highly engaging computer simulation game. The 
first study looked at relationships among tolerance for ambiguity, trait and state curiosity, and user beliefs and 
feelings and whether receiving information affected those motivation variables. The second study studied the effects 
of receiving information on several flow variables. The treatment differed from the first study in that it increased the 
amount of time allowed for subjects to be engaged in playing Myst and provided richer information to subjects in the 
information treatment group. 

Researchers have found positive correlations between flow and experimentation and exploratory behavior 
(e.g. Webster, Trevino, and Ryan, 1993) and the playfulness of flow in human-computer interactions has highly 
correlated with higher experimentation (e.g. Ghani, Supnick and Rooney, 1991). Research has also found that 
activities labeled as "play" rather than "work" positively influence intrinsic motivation and productivity (e.g 
Sandelands, 1988; Webster et al. 1990). Interfaces that incorporate motivation elements that enhance the playfulness 
of information systems may lead to greater exploratory and information seeking behaviors in users. 

Flow is largely perceived as positive for encouraging persistence at a task. However, Rafaeli (1985) and 
Jarvenpaa and Dickson (1988) argue that flow has a cost in terms of time and attention to a work task. People may 
be so engaged that too much time is spent on the task. Although these studies were conducted in "playful" 
situations, it would be worthwhile to implement similar research studies in work environments. In Study I, it 
appeared that interrupting a playful activity with information that could encourage more speedy resolution of the task 
may negatively influence motivation. However, in Study II, the information intervention had a somewhat positive 
effect on users' perceptions of control and did not appear to negatively influence flow.. 

Trevino and Webster (1992) recommend research to determine the relative effects of flow on productivity. It 
might also be useful to continue to study the effects of information interventions introduced at different points in the 
experience (beginning, middle) on mediating motivational factors such as curiosity and flow while learning a new 
software package or other engaging work task.Providing subjects with a choice of when to use an information 
intervention might also provide insights into when and how such interventions are useful. 
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Information Technology, Community, Place, and Presence 

Michael J. Streibel 
University of Wisconsin-Madison 



Abstract 

This paper argues for the importance of physical place and physical presence in how we construct meanings 
in our lives, how we form personal biographies and public histories , and how we develop living communities. This 
stance is contrasted with the consequences of the abstractions of place and presence in virtual, on-line communities. 

Introduction 

I am going to raise a number of questions about information technology which I hope will extend the 
discussion into areas that are important in our personal and public lives. The questions come out of two very 
different views within the public discourse about the role of information technology in our lives. Let me call one 
the technological vision and the other the experiential vision. 

The technological vision of life in the future is captured very well by Nicholas Negroponte in his book 
Being Digital when he says: 

Digital living will include less and less dependence upon being in a specific place at a 
specific time. (165) 

It is also captured by William J. Mitchell in his book City of Bits when he says: 

The virtual communities that networks bring together are often defined by common 
interests rather than by common location. (116) 

Where, literally, does digital living and virtual community really leave us? 

The experiential vision, on the other hand, is captured well by Thomas Moore in his book Care of the 
Soul, when he claims that: 

The soul prospers in an environment that is concrete, particular, and vernacular. (25) 

Soul here is not meant to be a metaphysical entity like a ghost in a machine but the quality of a relationship one has 
with particular people, places, and situations. I will, therefore, speak of the soulfulness of place in order to stay 
focused on the secular and vernacular. Later, I will also speak of spiritedness. 

These two vision of the future are clearly being put to the test by networked information technologies that 
virtualize people, places, and communities. I will also, therefore, try to stay focused on the importance of physical 
places and physical communities in our lives while always keeping an eye on the alternate claims of the 
technological vision. 

At this point, I want to remind you of a cartoon that recently appeared in the New Yorker because it 
captures many of the points I will make throughout this paper. In this cartoon, two dogs are facing a computer and 
the older dog says to the younger dog: “On the internet, nobody knows you’re a dog.” As you can see, the internet 
introduces something quite new into ordinary everyday communication. We can now carry on extensive interactions 
with other people without being fully present or even fully ourselves. In a way, this has always been the case with 
the use of written and printed communication, as well as with telephone, radio, and television communication. 
However, the internet takes these trends to new levels of abstraction because we now have: 

1. abstract representations of place and information (e.g., virtual libraries, virtual offices, virtual 
workgroups, MUDs, MOOs, virtual communities), 

2. abstract representations of self and other (e.g., text descriptions, avatars), and, 

3. abstract representations of interactions (e.g., intelligent tutoring systems, automated workflow 
processes). 

These trends will no doubt continue. However, I have a questions about these trends: what do we need to 
attend to as we move bravely into a cyber future? The answer is surprisingly simple: we need to attend to our 
physical bodies, our physical settings, and our physical communities. As the phenomenologist David Abrams says 
in his book The Spell of the Sensuous , the reason is that we are embedded in a physical matrix of human and more- 
than-human beings. Furthermore, within this physical matrix, as the deep ecologist George Sessions says in Deep 
Ecology for the 21st Century , meaningful work and meaningful life requires that we experience meaningful 
connections. Meaningful connections, in turn, as Dolores LaChapelle, another deep ecologist says in George 
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Sessions book and in Earth Wisdom , require an “intimate, conscious relationship with [a] place,” and “a stable, 
sustainable culture.”(57) I will address each of these separately before turning to computer networks and community. 

In summary for now, however, the choices we face have to do with coming to terms with the two 
tendencies of the technological and the experiential visions. Alan Drengson, in his study of The Practice of 
Technology , captures these tendencies when he says that: 

The industrial paradigm of development is based on monoculture, uniformity, speed, and 
abstraction, all of which run counter to the natural inclinations toward place-specific 
wisdom, diversity, accumulation of tradition and values, and sense of community, self- 
worth, integrity, value and meaning. (99) 

Physical Places 

I have a question for each of you. Think about the offices you are returning to after this conference. Think 
about the homes and apartments you are returning to later in the week. What do you want from these physical 
places? Normally, we do not have to ask these questions: living places and working places are two different places 
with their own rules and conditions. In our homes or apartments, we assume that we can arrange things in any way 
that we want in order to suit our desires. However, as William Mitchell says: 

The domestic living room is emerging as a major site at which digitally-displaced 
activities are recombining and regrounding themselves in the physical world. (98) 

Later, he goes on to say that our “buildings will become computer interfaces, and our computer interfaces will 
become buildings.” (105) Our living places in a digital world , in other words, are no longer solely sites for our 
inhabitation. 

In our work places, we have already conceded inhabitation. They are, after all, continually being redesigned 
to foster economic efficiency and productivity. However, even here we may be showing signs of wanting some 
measure of control over changes. That is, even though we accept automated manufacturing with computerized robots 
in our factories and automated business processes or administrative processes with networked computers in our 
offices, we still believe that the technology should be wrapped around us and not the other way around. Why is this? 

I am going to offer one of several possible answers: we are embodied in physical (and not virtual) bodies 
and we are situated in physical (and not virtual) locations. The fact that networked communications technologies 
allow our eyes and ears to sense information from other times and places ignores the fact that our lifeworld is 
situated in the physical here and now. We are, in effect, working here even though information is brought to us 
from elsewhere over networks. All important aspects of working and learning happen in the here and now in our 
physical bodies. Or, as Thomas Moore says in Care of the Soul : 

The love that goes out into our work comes back as love of self. Signs of this love and 
therefore of soul are feelings of attraction, desire, curiosity, involvement, passion, and 
loyalty in relation to our work. (187) 

Hence, even in our work world, we have to attend to our physical place and presence if we are going to have any of 
these feelings. 

Given these experiential facts, what can we learn from our physical bodies and from our physical places? 
Why are these questions even important? Moore again provides an answer: 

The body’s changes teach us about fate, time, nature, mortality, and character. Aging 
forces us to decide what is important in life. (216) 

The same can be said about physical places. The questions then become: do virtual bodies and virtual places teach 
any of us these things? Let me continue to answer these questions in a roundabout way. 

Christopher Day, one of the seminal architects of our time who wrote Places of the Soul , begins to answer 
these questions quite simply: we are deeply affected by the physical places where we live and work, and, our physical 
as well as psychological and social health depends on designing health-giving physical places. How can we do this 
if we are handed ready-made workplaces? According to Day, we can design health-giving physical places if we: 

1 . listen to what the physical place, the moment, and the community ask for, 

2. inhabit the physical places our bodies are in, and, 

3 . make our environment a part of our biography and history. 

I have summarized these ideas in the graphic below: 





Personal Space & Biography 




Figure 1 . Personal Space and Biography 



How many of you enter a new workplace or a new office and immediately put your stamp on the place? I 
suspect each of you has a unique story. In Day’s terms, you have begun to affect the ‘spirit of the place.’ Now, as I 
said earlier about soul, spirit is not some metaphysical substance but a quality or pattern of a place. That is why I 
use the word spiritedness rather than spirit. Other people who come by and engage you in your workplace will 
immediately recognize this new spiritedness. Furthermore, the more you work in a physical place, the more you 
infuse it with your spiritedness. The more others engage you, the more you co-create a public space. This, in turn, 
creates a common history that is tied to a specific place. I have summarized these ideas in another graphic below: 



Public Space & History 




Figure 2. Public Space and History 
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Let me take this line of inquiry one step further. How many of you have resisted strenuously when 
someone else tried to change these personal arrangements? I think we have all had this experience. Was this just 
personal ideosyncraticness? I think not. In Day’s terms, the place and its surroundings had in some small way 
become part of your biography and history. This is not to suggest that the place and its surroundings defined your 
identity but rather that the care which you infused into your environment changed your experience of the place and 
also changed other people's experience of you and your interaction with that place. These unique experiences are 
now part of your biographies and histories with these other people and places. Years from now, people will still 
remember the particularities of these times and places. The soulfulness and spiritedness of people and places are both 
sensed in real time and remembered by people. It binds them into a group of people that is more than a collection of 
individuals but rather an historical community. 

Can virtual places be infused with a similar soulfulness or spiritedness? Can virtual places go beyond 
being functional, economically efficient places? Before addressing these questions, let me expand the discussion 
briefly to include the notion of community because it contains some elements that will help us with our answers. 

Community and Place 

Why is it important to dwell in a physical place? Why not just keep our living places and our work places 
separate? Thoreau and Leopold come to mind here. They went off alone to separate physical places - Thoreau to 
Walden Pond and Leopold to Sand County, Wisconsin - and reported back what they learned. Their main insights 
were that their physical places were part of a larger community of interconnected places and that all human and non- 
human beings were part of a larger community of beings. Today, we refer to these two interconnections as ecology 
and Gaia. Abrams talks about how human beings are embedded with a more-than-human world. However, I would 
like to narrow my focus on the connections between human communities and places without losing sight of these 
larger connections. 

Daniel Kemmis, in his study of Community and the Politics of Place , explicitly connects citizenship with 
the inhabitation of physical places. 

Public life [he says] can only be reclaimed by understanding, and then practicing, its 
connection to real, identifiable places. (6) 

Kemmis further claims that since community and public life go together, community can only be realized in practice 
by people who have: 

1 . a living, working relationship to a physical place, 

2. a stake in each other’s lives, 

3. a commitment to human virtues such as neighborliness, 

4. a trusted pattern of behaviors, and, 

5 . an objectivity about what works and what does not work 
“No real culture,” he concludes, “can exist in abstraction from place.” (7) 

The question then becomes, can communities exist in virtual places? After all, since much of social reality 
is continually negotiated through communication, why not just extend communication to include 
telecommunications such as interactive video and the internet? However, what do the five points above mean if we 
extend the meaning of each term to encompass virtual interaction? I offer these questions as a beginning for further 
discussion: 

1 . Can one have a living relationship within a virtual place? We already see examples of having 
working relationships with virtual people and places. Living relationships, however, entail 
experiencing local patterns that connect one to larger patterns in the more-than-human world. 

2. Do we have a mutual stake in each other’s lives when the Other is at a virtual distance and 
only a representation? My tentative answer is that it gets harder and harder to have such a 
stake. 

3. Can we sustain the human virtue of neighborliness when the virtual medium permits anonymity and 
the possibility of continual disconnection? No! 

4. What trusted pattern of behaviors can conjointly be constructed with another person or place at 
a distance other than computational rules such as work-flow processes or stereotypical 
expectations? We would all have to operate according to the same abstractions for this to ever 
work in any sustainable way. 

5. How can we maintain a pragmatic objectivity when virtual worlds and places can be modified 
at will? I am not sure what pragmatism means in a totally constructed virtual environment. 
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Pragmatism seems a matter of predefinition in such virtual environments whereas it is not a 
matter of predefinition in environments of living relationships. 

All of these specific questions really come down to two different models of community. Following Kemmis and 
also Michael Sandel in his Democracy's Discontents , I will call these the engagement and procedural models. 

In a deep sense, the specific questions above and the two underlying models of community are really as old 
as our Republic itself because they appeared in the debates between Jefferson and Madison about what type of 
Constitution we really wanted. Jefferson believed that we should have a government through a politics of 
engagement at the local community level. Madison and the Federalists, on the other hand, believed that we should 
have a procedural Republic through a politics of radical disengagement. Jefferson, therefore, believed that people had 
to face each other in person, accept responsibility for their objective situations, and then work out common 
pragmatic solutions. Madison, on the other hand, wanted to keep people apart and use federal checks and balances 
and procedures to smooth the way for the growth of commerce. The Federalists won out in the writing of the 
Constitution. However, both sides realized that the debate would continue and so looked to the frontier and the 
creation of new States as an opportunity to realize their respective visions. 

Today, it seems as if the global communications networks in general and the internet in particular are the 
new frontier where we can escape from each other’s physical presence and avoid having to deal with the physical 
problems of communal living by forming virtual identities and virtual communities. The drive towards global 
economics still fuels these very developments. However, two things just don’t seem to go away, one old and one 
new. On the one hand, we still reside in our physical bodies, we still inhabit physical places, and we still have to 
get along with our physical communal neighbors. This may account for the continuing appeal of Jeffersonian 
democratic ideals. On the other hand, we have now reached the global limits of the earth in our attempt to 
commercialize people and places. At both the personal and the global levels, therefore, we now have to face the 
physical demands of people and places. 

Networks, Community, and Place 

Stephan Talbott, in his book The Future Does Not Compute , poses a provocative question: “can human 
ideals survive the internet?” (1) He then goes on to list the public relations benefits attributed to the internet: 

1 . “extended democracy, 

2. personal liberation, 

3 . enhanced powers of organization and coordination, 

4. renewal of community, 

5 . information transmuted into wisdom, 

6. education freed from the grip of pedagogical tyranny, 

7. a new and wonderous complexity arising out of chaos.” (1) 

The problem with the internet, however, is that it is “mostly a means for technical information exchange.” 
It is not a “means of solving the primacy of personal relationship and community in a depersonalized society.” (2) 
These latter things do not just happen because: 

every ideal worthy of the name can only be realized through some sort of conscious 
struggle and achievement. (3) 

Hence, developing community requires a “constant struggle and commitment,” a “charitable spirit,” and “confronting 
of self with other rather than hiding behind stereotypes and representations.” Here, we return to the Jeffersonian ideal 
of the politics of engagement in community building. 

Let me give you one specific example of a current struggle that can be seen from the two perspectives: the 
issue of privacy on the internet. Following Talbott, the engagement approach to privacy is based on: 

1 . interpersonal respect between human beings, 

2. a continuous struggle and commitment by community members, and, 

3. a constant reflection and understanding of the quality of human relationship as it relates to 
rights and responsibilities. 

The technological approach to privacy on the internet, on the other hand, is based on (5): 

1 . “technically-shielded anonymity”, 

2. “unaware and unthinking others unless the technological walls [passwords and firewalls] are 
broken”, 

3. “legal sanctions as protection”, and, 

4. “constant technological improvements and countermeasures”. 
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As you can see, the former requires that people be fully present and committed to each other. The latter allows 
people to “retreat into communal abstractions.” (140) Talbot, therefore, concludes that the biggest danger of the 
internet is that it “deemphasi[zes] ... people who are fully present.” (140) This, in turn, undermines community. 

Conclusion 

I want to end this discussion with a series of questions and some tentative, personal answers. The global 
internet phenomenon is so recent, swift, and total that questions and tentative answers are the best way to sort out all 
of the issues. I invite you to participate by offering your own tentative answers. In giving these personal answers, I 
will try to make the case for the importance of caring for our own physical presence, our physical settings, and our 
physical communities. 

Question 1: Why is it important to spend each day connected to and caring for our physical bodies and the 

physical places where we live? 

My own tentative answers are because: 

1 . disconnection of physical self and place could lead to disconnection from others and a subsequent loss 
of communal life which I value. 

2. daily reconnection reinvigorates being an inclusive subject in my life rather than being an object in 
someone else’s life. 

Question! 2: What happens when the bulk of our daily experiences, work, and learning takes place in 

virtualized, on-line communities? 

My own tentative answers are that in virtual communities: 

1. I begin to interact with computationally-mediated intermediaries and may begin to see myself and 
others in similar terms. This is not how I want to see myself or others. 

2. all local particularities of person, place, and situation fade into the background. This is not how I want 
to experience the world. I actually want every local experience to connect me in some small way to the 
larger patterns of the more-than-human world. 

3. what is communicated by others is an idealized, objectified, computationally-represented pattern of 
information rather than a fully sensuous, material being. Such a being entails the full range of 
intelligences described by Howard Gardner in Frames of Mind and all the patterns of embodied, natural 
intelligence described by Donna Markova in The Open Mind. 

Question 3: Why do we need to deal with people face-to-face rather than at a virtual distance? 

My tentative answer is that I want to notice and interact with people daily in order to: 

1 . gain greater understanding of myself as a social being , and, 

2. co-create a lifeworld with other people in specific places. 

Question 4: Why should we open ourselves up to the experience of our physicality and our physical 

situatedness in the here and now? 

My tentative answer is best described by David Abrams in his study of the Spell of the Sensuous (1996) 
where he said that: 

the world and I reciprocate one another [such that] ... our spontaneous experience of the 
world [is] charged with subjective, emotional, and intuitive content [and] remains the vital 
foreground of all our objectivity. (33-34) [my empahsis] 

These experiences then become the grounds of my being and, through empathy, the basis of my recognizing others 
as subjects. The experience of self as embodied-subject and other-as-embodied-subject leads to the experience of 
community-as-embodied-subject and even to the experience of world-as embodied- subject. All of these together are 
my main source for objectivity and my arena for the development of pragmatism. 

Question 5: Why should we develop a physically-present community before we extend it with virtual 

communications technologies? 

My tentative answers are that: 

1 . my immediate lived experience will always be prior to my thoughts about it. 

2. If I do choose to ignore my lived experience, I might be denying myself the ability to develop new 
concepts and understandings. 

My tentative answers here are not meant to suggest that we should deny the value or practice of virtual communities 
but rather that we should give precedence to our physical community, place, and presence in our lives. 
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These questions and tentative answers are really my way of continuing an inquiry. In a 
strange and ironic way, the internet phenomenon has forced me to ask questions about my physical 
self, my physical setting, and my physical community which I previously took for granted. 

I will end this paper with a few lines from T.S. Eliot’s poem Little Gidding: 

We shall not cease from exploration 
And the end of all our exploring 
Will be to arrive where we started 
And know the place for the first time. 
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Understanding the Design and Use of Learning Technologies 
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For the past few years, I have grown increasingly interested in technology adoption. Why do people choose 
to make use of an instructional product or learning resource — and why do they sometimes resist? My interest in this 
question has been stimulated by witnessing an accumulating number of failed instructional design (ID) projects — 
failed in the sense that products were weakly or only partially implemented in the way intended by their designers. 
Of course, the failure of ID projects is nothing new; war stories of such failures are part of every experienced 
designer's repertory of cautionary tales. Yet remarkably little discussion of lessons learned from failures is made in 
the literature. We seem to be much more willing to discuss our successes than our flops! 

Adoption concerns are increasingly important to understanding constructivist learning environments and 
learning communities. As instruction begins to move toward more open, participatory models, "end users" — both 
teachers and learners — are asked to take more responsibility in the learning process. Seen as adopters of technologies 
and products, participants need to be sold on the approach and supported in their new learning. 

Another accounting for my renewed interest in adoption lies in the different responses to the Internet and the 
World-Wide Web. As an observer, I have witnessed widely varying reactions to this overwhelming new technology, 
ranging from populist enthusiasm to mistrustful resistance. People differ widely on questions of free speech, 
copyright, Web publishing, and the learning potential of the Web. Some people are ecstatic that educational 
institutions have lost their near-monopoly on learning resources; others fear that inaccurate, unreliable information, 
coupled with uncontrolled communication, will lead to a number of social and educational problems. Some people 
depend on e-mail not just for professional communication, but as a vehicle for intimate sharing of deeply human 
experiences. Others have decided that e-mail intrudes too heavily into their lives; that in sum, the impact on quality 
of life is too great to be worth the benefits. 

These issues — the variable implementation of ID products, the functioning of open models of instruction, 
and the pros and cons of the Internet — have a direct impact on my life, since I am a designer of learning 
environments and an active participant of Internet resources. As a result, I have been relating these newer adoption 
issues to traditional concepts of instructional design I was trained on. How does design relate to use? Are the two 
spheres competing or complementary? Do they overlap or are they separate concerns? 

The purpose of this paper is to explore some of these issues. To understand how adoption relates to design, 
I discuss both concepts together, with implications for both design and adoption practices. My hope is that we 
move toward a reform of professional practice, with stronger links to contexts of use, and correspondingly greater 
attention to adoption and effective use of learning resources. 

Use Concerns within the Design Process 

Instructional design involves the preparation, design, and production of learning materials. The ID process 
results in several key instructional components, including: 

— learning goals and objectives; 

— methods and instrument for assessing learning progress; 

— content or information needed to accomplish the learning objectives; 

— messages to be presented; 

— student activities and learning interactions. 

Some of these can be more completely prespecified than others, and the degree of prespecification will vary. For 
example, computer-based tutorials will need to be more pre-packaged than classroom resources for an experienced 
teacher. 

Questions of use and adoption impinge directly into the design process. The products of ID — instructional 
resources — are meant to be used, either by a particular individual or group or a more general audience. Products are 
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worthless if they are not used. With this in mind, the ID process includes a number of strategies or steps to 
accommodate the likely needs of users, most notably 
— context or environment analysis (Tessmer & Harris, 1992); 

— formative evaluation to provide data from field tryouts and user feedback. 

Efforts receiving somewhat less emphasis but equally important are participatory design, that is, the 
inclusion of user/practitioners throughout the design process; and consensus-building among users and 
constituencies. 

These strategies are "use-oriented"; they increase the likelihood of successful implementation because they 
take the end use into account at beginning design stages. For that very reason, however, such strategies are often 
neglected at the expensive of more immediate, pressing needs of design and production. Involving end users in the 
design process is often seen as an unjustified expense. Formative evaluation and related strategies seem to be 
honored more often in breach than practice (Wedman & Tessmer, 1993). 

So why are so many carefully designed products sitting on shelves? Perhaps designers don't incorporate 
enough of these use-oriented strategies. Or perhaps the overall quality of the design is flawed. I have heard design 
proponents argue that if you do your analysis right up front, your product will fill a legitimate need; ergo, it will be 
used. Somehow, though, I am not convinced that careful design alone can ensure successful implementation and 
use. We need renewed attention to utilization concerns throughout the design process, through strategies grounded in 
new theory. We also need to better understand the adoption process as a separate sphere of concern. 

Roles versus Activities of Design and Use 

I find it useful to distinguish primary roles from functions or activities. My primary role may be that of an 
instructional designer, yet I may engage in various activities where I try out or use the product or service in question, 
and participate in the community of practitioners and users. Similarly, an end user may participate in the formal 
design process, or may engage in design-type activities on the job — after the product’s design is allegedly complete. 
Table 1 depicts this crossing-over effect. 

This distinction acknowledges the mixed nature of the work of both designers and end users. To be 
effective, designers must participate in the practitioner culture to at least some degree regardless of their level of 
content knowledge. Similarly, to make best use of learning resources, end users must think in design terms, 
appropriating those resources and making them fit local conditions. 

Learning Resources: Artifacts or Offerings? 

The products of instructional design can be thought of in a number of ways. At one level, they serve as 
knowledge containers, carrying the expertise of designers and their backers, intended to convey that expertise to 
students. In this role, learning resources become artifacts which support individuals and groups in their knowledge- 
based activities. Knowledge is stored, in other words, in people’s heads, in a culture’s shared beliefs and values, and 
in that culture's repository of artifacts — documents, products, institutions, and so forth. Instructional materials 
constitute an important kind of artifact in our culture. 

Seen as knowledge containers, learning resources embody expertise, yet they cannot stand alone. Of 
themselves, they have no knowledge value. Only in their appropriation and use by people can their knowledge value 
be realized. 

At another level, learning resources can be seen as tools in the hands of an end user, enabling a teacher to 
effectively communicate, present information and direct fruitful interactions. Students, too, can appropriate learning 
resources in tool-like fashion, to further their learning according to their interests and goals. 

The tool metaphor stresses the enabling or facilitative function of learning resources. A product is thought 
to be useful as it is manipulated to solve someone’s problem. Using the tool, teachers and students are able to 
communicate, teach, and learn more effectively. 
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Designers 



ROLE 



End users 



Traditional ID 
activities — plannin g, 
analyzing, designing, 
etc. 


Field experience 

Practitioner 

experience 

Practitioner dialogue 

Community 

participation 


Adapting instruction 
to locating setting 

Integrating technology 
into curriculum 

Selecting parts of 
product to appropriate 


Traditional 
utilization and 
adoption 

activities — teaching 
the lesson, 
completing the 
assignment, etc. 



Design Use 

ACTIVITY 

Table 1. Overlap between roles and activities among instructional designers and users of their products. 

A more personal way to see learning resources comes from asking, who is communicating what to whom? 
Learning resources are presented to students within a social context, and that web of social relationships helps to 
define how students interpret resources. Suppose a 7th grade girl is given a self-paced book meant to teach grammar. 
How might this student respond to the proffered learning resource? A number of possibilities are open to her, 
including: 

— accept or reject the book; 

— think about what’s being read or plow through to complete the task; 

— actively question or passively receive the content; 

— relate the book to prior knowledge or tack on top of prior knowledge; 

— upon perceiving the book's weaknesses, quit; or overlook the weaknesses, compensate, and continue on. 

Part of the student’s response to the book will depend on who gave her the book. Does she trust the teacher 
to seek her best interests? Does she think the teacher likes her as a person? Does she associate the book and the 
book-giver with positive experiences in her life? The point of view presented in the book itself might also affect her 
response. Do she and the author connect? Does the author consider her prior knowledge and communicate new ideas 
clearly and effectively? Asking about the learning resource in these ways treats it as something offered to the student 
by another person. The person-to-person relationships have a bearing on the acceptance of the resource. 

When instruction is seen as an extension or offering of another person, the student enters into a kind of 
relationship with that person through the instruction. The designer of the materials might be heard saying, 
"Whoever you are out there, I am trying to reach you and meet your needs.” The student might respond to the 
resource — or the gift- giver associated with the resource — "You are asking me to read, or think, or answer a question. 
Do I trust you to teach me something worthwhile if I cooperate?" Or, "You want me to learn from this book, but 
it’s so boring! You're not listening to my needs!" 

On the other hand, the student might not think in such terms at all. In a recent conversation with my son 
over a poor grade in English, I suggested that his teacher might be disappointed in his performance. "What? What 
are you talking about?" He had never contemplated the possibility that when students refuse to hard work and learn, 
a teacher might feel bad. Teachers are people too, I reassured him! 

Whether or not students personify the intentions of the teacher or implicit author, they nonetheless can be 
said to engage in a kind of dialogue with the instruction. There is a mutual responding back and forth between 
teachers and students, or between learning resources and users of those resources. The conversation is an exchange of 
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ideas and thoughts, based on a relationship with a particular history of trust, varied motives, mistakes, and 
forgiveness. 

Reading, Writing, and the Web 

At this point, I feel rather out on a limb. This kind of language — trust, relationship, forgiveness — just 
isn't found in the ID or technology literature. What value is there in speaking this way? As we mentioned at the 
outset, as learning resources find their way into open learning environments and communities, students are expected 
not just to perform, but to make wise decisions. For an open learning system to prosper, participants must choose 
to use learning resources to support their own learning and for the growth of the whole group. Students in such 
environments can still look to teachers and informal leaders for direction, but responsibility is much more diffused 
than with traditional teacher- led instruction. Under such conditions, questions of motivation become at least as 
important as questions of technical skill-building or knowledge accumulation. And motivation, I am increasingly 
convinced, can only be understood by including cultural and social factors, to supplement traditional psychological 
and information-processing factors. 

The distinction between design and use becomes most blurred in open hypertext environments like the 
Internet. George Landow (1992) has explored how hypertext turns readers into authors by giving them control over 
the story line, the flow, the content, and ultimately the meaning of the text. Similarly, hypertext authors surrender 
control and participate more flexibly in the conversation, reducing the traditional gap between reader and author, in 
the discussion. 

What is true for literary criticism is true for education as well. I send my students to the lab to browse the 
Web and complete a research assignment. Who is the instructor here? Who determines content, sequence, and 
learning objectives? More than before, I share the design function with my students. And that is just the beginning. 
When my students go home or stay after school, browsing the Web and initiating their own learning activities, they 
have appropriated much more of the design function away from formal institutions. That such activity is going 
on — among both adults and children — is evident from the tremendous growth of listservs and online discussion 
groups of all kinds. The informal learning happening is nothing new; what is new is the technology that allows 
powerful representations and communications to take place, and the resulting burst of human knowledge now being 
found on the Web. 

Continued Blurring of Distinctions 

We have seen a parallel between: 

— designers and end users; versus 
— teachers and students. 

With both pairs, the role should not be confused with activities or functions being performed. Designers 
can be users, just as teachers can be learners. Also, both relationships are amenable to a wide range of negotiated 
roles. We can build systems where the line between designers and users is entirely blurred, with mixing of roles and 
crossing over of assignments. This is presently the case on the Internet, and more particularly, in MUDs and 
MOOs, where users can become designers almost from the start, and where the environment's design benefits and 
suffers accordingly. Open models like this contrast strikingly with traditional institutions of learning. The same 
contrast is seen among teachers and students — Some models throw everyone into an environment, with barely 
discernible differences in roles. In general, we might say that models with highly contrasting roles are more top- 
down, hierarchical, and formal, whereas models with merging of roles are more open, decentralized, and informal. 

Which is better? Should we maintain role distinctions between the expert authority and the end user, or 
should we encourage the merging of responsibilities? Under what conditions would we expect clear role divisions to 
be helpful or hurtful? These questions can be addressed from different perspectives — scientific, political, moral. A 
static, scientific approach would tend to look for general rules that govern such systems. A more contextualized 
approach would be to look at the history of interactions and the relationships between actual participants. What has 
been negotiated in the past? What kind of local culture has evolved? What are people's expectations? 

Consider a classroom example. A 10th grade boy asks to have a biology assignment waived, because he 
already understands the concepts of the coming unit, and he wants to work independently in a related but separate 
direction. In essence, the student is asking to assume more of the teaching role himself, to create his own agenda 
and take charge of his own learning. The teacher responds to this request based upon the relationship between the 
two of them. The student has completed assignments punctually and has performed well on exams and labs. He is 
old enough to have developed mature study habits. On the other hand, customizing a plan of study will require more 



supervision and vigilance on the teacher's part. Making an exception could also set a bad precedent for other students 
less ready to handle the responsibility. When it comes down to it, the decision is negotiated between the two of 
them, based upon these factors and a variety of others. In large part, the decision rests upon the teacher's willingness 
to accommodate individual needs, and reapportion additional energy and attention toward the individual student. The 
outcome depends heavily on contextual issues, local and unique to the situation and not easily subsumed by general 
rules and principles. 

In summary, role distinctions generally serve a valuable purpose. Access to expertise in the form of a 
teacher or well-designed instructional materials can spare learners headaches and wasted time. In an open market, 
people spend enormous sums on formal training and instructional products, because they have a hard time learning 
by themselves. At the same time, emerging network and representation technologies threaten to displace the 
designer/teacher's near-monopoly on learning support. We are led, then, to an irony: Designers and teachers are 
most effective when they participate along with their clients in the practitioner culture and reach out to contexts of 
use; similarly, end users and students perform best when they assume more responsibility for their own actions and 
engage in designing/self-teaching activities themselves. 

Implications for Practice 

A co-dependent view of designing and using technologies leads to a number of implications for practice. 
For instructional designers, several conclusions can be drawn: 

ID is much more than materials- or message design. Seen in the context of practice, ID incorporates issues 
of utilization, market, and adoption. The more practitioners enter into the design process, the greater likelihood that 
the outcomes will meet the needs of the field. Generally, these concerns tend to be neglected, both in practice and in 
ID literature. 

Consideration of context of use is more than adding steps to a design model. ID will best be served by a 
more fundamental shift in perspective, granting equal status and concern to issues of adoption and use. The most 
recent edition of Dick and Carey (1996) contains a new section on context analysis, filling twenty-six of more than 
three hundred pages. This is good, but it's only a start. Designers who have any hope to seeing their work used 
effectively need to respond more closely to the needs of their clients. This cannot be ensured by a front-end 
questionnaire or reliance on a subject-matter expert pulled away from her job. There needs to be more of a meeting 
between design lab and job setting, where participants interact and participate in a common culture. 

Participatory design is a somewhat redundant term. End users always function as designers as they 
appropriate and use learning resources, even if they are left entirely out of the formal design of those learning 
resources. Like it or not, users continue the design process as they determine how and where to make place for 
resources in their lives. 

' Teacher-proofing " materials is impossible. The kind of attitude that says, "use these materials as we have 
designed, and don't deviate from our plans" only serves to weaken the value of those materials. Rigid, inflexible 
products that block users from experimenting or adapting can be frustrating and off-putting to users. Such products 
are crying out to be ignored. Our point is that teachers must appropriate and adapt materials to their local 
conditions; in doing so, as we have emphasized, they are co-designers of the learning experience, along with their 
students. 

The same negotiated dynamics between designers and teachers exist between teachers and students. Teachers 
and students depend upon each other to cover for one another — students filling in what the teacher missed, and the 
teacher learning along with the students. In this way, they mirror the relationship between designer and user. No 
designer can know from the beginning just now a product will be put to use. A healthy respect for varied contexts 
of use will result in learning resources that are more flexible, modular, and accessible. 

Designers must look for total effects of interventions on all participants. Many innovations fail because 
they neglect the changed roles demanded of the people expected to use the technology. User participation in design is 
a step toward accommodation of their perspectives. Designers are well-advised to carefully analyze the expectations 
placed on participants, and design ways of supporting those participants in their new roles and activities. 

There are additional implications extending to adoption and use activities, including the following: 

End users need to think like designers as well as consumers. Teachers need to be taught to actively 
appropriate resources and technologies, rather than passively follow the program. Curriculum design and integration 
may begin at national or district levels, but the most important work is done at the local level. Teachers need 
permission and validation to complete the task. 



We need new frameworks for understanding adoption and change. Present discussions of adoption are 
heavily influenced by the Everett Rogers (1995) model of diffusion of innovations. This model is descriptive of 
objective categories and types, but tends to neglect underlying systemic processes. The present paper has emphasized 
adoption perspectives "from the inside"; that is, from participants’ points of view. Systems models can complement 
that inside view by providing a useful outside view ( cf Wilber, 1996), showing how individuals and groups 
accommodate new technologies and innovations, following predictable patterns of resistance, accommodation, and 
integration. 
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Abstract 

The purpose of this study was to investigate whether color and background information in motion visuals 
have different effects on memory and comprehension of students in various grade levels. A 3 (levels of visual 
complexity) X 3 (grade levels) factorial pretest-posttest design was used. The instructional content was a 12- 
minute computer animation concerning life styles of animals. A total of 412 third , sixth and eighth graders was 
randomly assigned to one of the three treatment groups. They were simple group , simple with color group , and 
simple with color and background information group. All subjects received a pretest , treatment , and two 
posttests . The posttests were designed to test subject *s memory and comprehension of the instructional content 
respectively. In addition , a questionnaire was also administered to the subjects in order to understand their 
perceptions of visual complexity in the treatment . The results of this study indicated that there was interaction 
between amount of visual complexity in motion visuals and grade level. Color and background information 
would affect the third graders’ achievement in memory and comprehension differently , while both the sixth and 
eighth graders’ performance were not affected by the amount of visual complexity in motion visuals at all. 
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Introduction 



With the enormous utilization of visual communications in learning environments, today visual messages 
have become an important medium, besides verbal and written language, for communicating instructional 
information to students. However, many researchers of visual instruction found that the use of visuals was not 
a panacea for improving student achievement (Dwyer, 1978; Levie & Lentz, 1982; Peeck, 1987). The various 
visual variables play a significant role in student's ability to memorize and comprehend the instructional 
materials. 

There are two attributes in visual complexity which have been recognized by researchers as important 
factors influencing learner’s processing of visual information. They are color and background information 
(Dwyer, 1978; Fleming, 1987; Tullis, 1981; Jones, 1989; Franken, 1977). The environment in which we live 
is colorful. Consequently, most visuals used for instructional purposes is now in color. The uses of color in a 
visual can be decorative, affective, attentional and cognitive (Pettersson, 1989; Pruisner, 1993). However, 
there has been considerable debate about the effectiveness of color as an attribute of visual information. Tullis 
(1981) indicated that the effectiveness of color was highly dependent on the task for which it was used. He 
concluded that color could be beneficial to performance, but no more so than shape coding. Livingston (1991) 




also found that color appeared to be a distracting variable because learners could often remembered the color of a 
hidden object while forgetting what the object was itself. On the other hand, Pruisner (1993) reported that the 
use of color had a positive impact on the recall of verbal information presented in graphic form. After 
conducting over 100 studies in this area, Dwyer (1982-83) also concluded that for the specific types of 
educational objectives the use of color had been found to be an important variable for improving student 
achievement. However, whether effects of color in instructional materials interact with learner age and 
cognitive levels is still unanswered. Thus, research into color as an instructional attribute is far from 
conclusive. 

The influence of background information on cognitive learning is another issue examined in the research 
on visual complexity. Many researchers have indicated that perception of a visual is determined not only by its 
characteristics but also by its surrounding context (Fleming, 1987; Antes & Metzger, 1980; Pettersson, 1989). 
A number of empirical investigations have focused on the presence or absence of background information in 
visuals. One of the earliest studies was conducted by Spaulding (1956). He found that visual information 
unnecessary to critical figures should be eliminated because it may motivate an interpretation that was not 
compatible with the purpose of the illustration. However, Antes & Metzger (1980) indicated that the context of 
objects helped learners construct a general characterization of the pictures which provided learner expectancies 
to perform their discrimination task. In general, the research results in this area are inconclusive and usually 
contradict one another . 

Furthermore, many studies also indicate that there are developmental changes associated with how 
children perceive visuals of different amount of complexity (Pettersson, 1989; Collins, 1970; Miller & Burton, 
1994). Although the research results appear inconsistent to some extent, researchers agree that there are age 
differences in how the children allocate their attention to a visual task. In addition, according to the findings of 
prior studies (Chen, 1993; Hozaki, 1988), there is an indication that humans may employ different internal 
cognitive processes for memorizing than for comprehending a visual display. However, as Shuell (1986) has 
indicated , it is not clear how these two mental processes are different. Does the visual information play 
dissimilar roles during these processes? It is evident that more research has to be done before we can draw a 
general conclusion on these issues. 

Another limitation of the prior research concerning the effects of visual complexity on cognitive learning 
is the static visual stimuli used as the material to be memorized or comprehended. Only a few studies have 
attempted to investigate the relative effectiveness of motion visuals that employ different levels of visual 
complexity to complement verbal instruction (Stone, 1983; Acker & Klein, 1986; Chen, 1993). In view of the 
increasing use of motion visuals in school settings, there is no doubt that more research work needs to be 
conducted regarding how students acquire learning information from moving visuals. 

In summary, for lack of thorough and conclusive empirical evidence concerning effects of visual 
complexity on cognitive learning tasks across age levels, teachers and instructional designers usually rely on 
their intuition in deciding how to select and present visuals in instructional materials. Therefore, it is evident 
that there is need to further investigate this area. 
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Research Questions 

The purpose of this study was to investigate whether differences in the level of visual complexity in motion 
visuals have different effects on memory and comprehension of students in various grade levels. Specific 
questions related to the problems were as follows: 

1. Does the addition of colors in motion visuals have an effect on student s' ability to memorize and 
comprehend the instructional materials? 

2. Does the addition of background information in motion visuals have an effect on students' ability to 
memorize and comprehend the instructional materials? 

3. Does the effect of visual complexity in motion visuals relate to students' age? 

4. Does visual complexity in motion visuals play different roles in the processes of memorizing and 
comprehending? 



Methods 

Subjects 

Subjects for the study consisted of 412 third (N=T31), sixth (N=T33) and eighth (N=148) graders from 
three public schools in Taiwan. They all were average students and were randomly chosen by school curriculum 
coordinators. 

Instructional Content 

The instructional content for this study was a 12-minute computer animation concerning life styles of 
animals, such as migration, classification, protective coloring , running speed. The main reason for selecting 
this topic was that movement was a defining attribute of the concept taught in the lesson. Furthermore, in 
order to examine the subjects' both memory and comprehension achievement, the instructional content was not 
presented through the method of direct instruction. Instead, it was displayed through a series of short events 
happened during a swallow's journey. 

Instructional Treatment 

The treatments for this study were created using the Animator Pro program that runs on a PC environment. 
There were three versions of color computer animation representing three levels of visual complexity for this 
study. They were outline drawing (OD), outline drawing with color (ODC), and outline drawing with color 
and background information (ODCB). 

In the OD version, the computer animation retained only the outline shapes of critical objects necessary to 
convey the main meaning of the narration (see Figure 1). The ODC treatment was the one in which different 
colors were added to the outline shapes of the first version (see Figure 2), and the third was the version in which 
both colors and background information were added (see Figure 3). Although the three versions of the 
treatment had different levels of visual complexity, they all employed the identical narration to complement the 
visual information. 



A31 




Figure 1. A Sample Frame in Outline Drawings (OD) 
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Figure 2. A Sample Frame in Outline Drawings with Color (ODC) 




Figure 3. A Sample Frame in Outline Drawings with Color and Background Information (ODC) 
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Instrumentation 

There were four instruments used in this study. They were pretest, memory test, comprehension test, and 
questionnaire. The pretest consisted of only 10 multiple-choice questions which tested students' prior 
knowledge about the instructional content. The memory test (KR20= .80) was designed to test students 
memory of content which was presented in the computer animation. There were 33 multiple-choice questions 
in this test. The comprehension test (KR20= .85) was composed of 25 multiple-choice items which measured 
students’ comprehension of instructional content. The fourth instrument was the questionnaire used for 
obtaining students’ perceptions of the computer animation. The questionnaire contained 3 open-ended 
questions, such as did you think whether the screen of the computer animation would be to boring or too 
complex; did you want to continue watching the animation? Why; of the whole story, which part did von 
remember most. 

Procedure 

The subjects of third , sixth and eighth graders were randomly assigned to one of the three treatments 
respectively. Then they received a pretest to determine their prior knowledge level in the instructional content. 
However, since the possibility existed that the pretest might activate student's attention toward some specific 
areas of the computer animation, the pretest was administered four weeks before the experimental treatments 
were delivered. 

After four weeks, subjects received their respective experimental treatments and two posttests (memory 
and comprehension tests). Upon the completion of the experiment, several students from each experimental 
group were randomly selected to answer the open-ended questions in the questionnaire. The researchers and 
two assistants encouraged these subjects to express their opinions to the computer animation and write these 
responses down in the questionnaire. 

Results 

The design of the study was a 3 (levels of visual complexity) X 3 (grade levels) factorial pretest-posttest 
design. In order to determine whether there were existing group differences in achievement levels of subjects in 
different treatments, an ANOVA procedure was utilized. The results showed a nonsignificant difference. 
F(2.403)= .06, p=,9427, for pretest scores among the three treatment groups. Therefore, the statistical results 
for this study were analyzed using ANOVA for main effects and their interactions. In addition, a post hoc 
Scheffe method was used to test for differences between pairs of means. 

Analysis of Memory Test Results 

The memory test means and standard deviations for the three treatment groups by grade levels were 
presented in Table 1. Summary ANOVA statistics were shown in Table 2. As shown in Table 2. there were 
significant differences for the main effect of grade and interaction effect of treatment and grade. Therefore, the 
simple main effects were further examined and the results reported in Table 3. According to Table 3. three 
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obtained F ratios were significant. They were the 3rd grade in treatment conditions as well as the OD and 
ODCB groups in grade levels. 



Table L Memory Test Means and Standard Deviations by Treatment Condition for Grade 3, 6, and 8 







OD 




ODC 




ODCB 




N 


M SD 


N 


M SD 


N 


M SD 


3rd Grade 


45 


21.69 5.96 


43 


25.37 4.24 


45 


22.93 4.99 


6th Grade 


44 


27.00 3.63 


44 


25.68 4.78 


43 


2514 411 


8th Grade 


49 


25.96 3.87 


48 


25.88 4.60 


51 


27.43 2.57 


Table 2. Analysis of Variance of Treatment Condition and Grade Level for Memory Test 


Source 




ss 


DF 


MS 


F 


P 


Treatment 




40.11 


2 


20.06 


1.05 


0.3521 


Grade 




754.70 


2 


377.35 


19.69** 


0.0001 


Treatment x Grade 




427.50 


4 


106.87 


5.58** 


0.0002 


Error 




7722.88 


403 


19 16 






**P<01 














Table 3. Analysis of Variance of Simple Main Effect for Memory Test 






Source 




ss 


DF 


MS 


F 


p 


Treatment 














3rd Grade 




323.64 


2 


161.82 


8.44** 


0.000 


6th Grade 




81.00 


2 


40.50 


2.11 


0.122 


8th Grade 




82.87 


2 


41.44 


2.16 


0.116 


Grade 














OD 




711.41 


2 


355.71 


18.56** 


0.000 


ODC 




6.76 


2 


3.38 


0.18 


0.838 


ODCB 




483.93 


2 


241.97 


12.63** 


0.000 


Error 




7722.88 


403 


19.16 







**P<01 
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Since the number of subjects in each treatment groups were different, the Scheffe method was used to test 
for differences between pairs of means. The results were as following: 

1. In the third grade , the ODC treatment surpassed the OD and ODCB treatments. As for the sixth and 
eighth grades, the three treatment groups performed in similar ways. 
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2. For the OD treatment, the third graders' performance was inferior than the sixth and eighth graders' 
performance. For the ODC treatment, there was no significant difference among grade levels. The 
eighth graders in the ODCB treatment exceeded other two grade levels. 

Analysis of Comprehension Test Results 

The comprehension test means and standard deviations for the three treatment groups by grade levels were 
presented in Table 4. Summary ANOVA statistics were shown in Table 5. As shown in Table 5, there were 
significant differences for the main effect of grade and interaction effect of treatment and grade. Therefore, the 
simple main effects were further examined and the results reported in Table 6. According to Table 6, three 
obtained F ratios were significant. They were the 3rd grade in treatment conditions as well as the OD and 
ODCB groups in grade levels. 



Table 4. Comprehension Test Mean and Standard Deviation by Treatment Condition for Grade 3, 6 , and 8 





OD 




ODC 




ODCB 




N 


Mean 


SD 


N 


Mean 


SD 


N Mean SD 


3rd Grade 


45 


12.44 


5.24 


43 


15.49 


5.57 


45 15.02 4.93 


6th Grade 


44 


18.30 


4.39 


44 


17.07 


5.50 


43 16.42 5.03 


8th Grade 


49 


18.41 


4.48 


48 


17.65 


4.92 


51 19.04 3.61 


Table 5. Analysis of Variance of Treatment Condition and Grade Level for Comprehension Test 


Source 




ss 


DF 




MS 


F 


P 


Treatment 




15.11 


2 




7.55 


0.32 


0.7269 


Grade 




1204.97 


2 




602.48 


25.47** 


0.0001 


Treatment x Grade 


349.89 


4 




87.47 


3.70** 


0.0057 


Error 




9533.99 


403 




23.66 






**P<01 
















Table 6. Analysis of Variance of Simple Main Effect fo 


r Comprehension Test 




Source 




ss 


DF 




MS 


F 


P 


Treatment 
















3rd Grade 




250.70 


2 




125.35 


5.30** 


0.005 


6th Grade 




80.17 


2 




40.08 


1.69 


0.185 


8th Grade 




55.22 


2 




27.61 


1.17 


0.312 


Grade 
















OD 




1056.03 


2 




528.02 


22.32** 


0.000 


ODC 




111.99 


2 




56.00 


2.37 


0.095 
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ODCB 407.93 2 203.96 8.62** 0.000 

Error 9533.99 403 1916 

**P<01 



Since significant F ratios were found, differences between pairs of means were analyzed via Scheffe 
procedure. The results were as following: 

1. In the third grade , the OD treatment performed worst among the three treatments, while the performance 
of the ODC and ODCB treatments was no different. In the sixth and eighth grades, the three treatment 
groups performed in similar ways. 

2. For the OD treatment, the third graders' comprehension performance was the worst among the three grade 
levels. For the ODC treatment, there was no significant difference among grade levels. The eighth 
graders in the ODCB treatment exceeded substantially the third and sixth graders. 



Analysis of Questionnaire 

An analysis of the data from the questionnaire revealed the following findings: 

1. Most students in each grades liked the computer animation because it was presented through a story style. 

2. The OD groups at each grade levels perceived the level of visual complexity in the computer animation was 
too dull. The reason why students wanted to continue watching the animation was that they would like to 
know more about animals they were not quite familiar with yet. 

3. Third graders preferred the ODC version and indicated that the visuals in the ODCB version were too 
complex. However, the sixth and eighth graders prefer visuals with background information and 
indicated that the visual complexity and color in the ODCB were just right. 

4. The instructional contents of the computer animation remembered most by the students was the knowledge 
they did not possess before, such as a giraffe having four stomachs. 

Discussion 




Level of Visual Complexity in Motion Visuals and Grade Level 

The results of this study indicated that children by sixth grade had acquired the ability to selectively 
process visual information. This finding was different from some earlier research which had previously 
reported that only until adolescence did children show the efficient use of selective attention to a learning task 
(Collins, 1970; Hale & Taweel, 1974; Hagen, 1972). A possible reason accounting for the developmental 
advance may have been that with the increasing impact of visual messages in our society, visual instruction has 
become one of the major methods of communicating information. Therefore, under such a learning 
environment, children capable of selectively allocating their attention were younger than ones in the 1970s. 
This finding verified what Hurt (1991) has stated that the amount of experience children brought to would affect 
them interpreting visual information. 
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Furthermore, the post hoc analysis of this study indicated that third-grade students in the OD group 
performed much worse on both memory and comprehension tests than students in sixth- and eighth-grade, while 
in the ODC version, the three age groups performed equally well. In term of the levels of visual complexity, 
the third graders’ memory performance in the ODC treatment surpassed in the OD and ODCB treatments, their 
comprehension performance in the ODC and ODCB treatments exceeded in the OD treatment. Thus, although 
the prior research suggested a visual presentation should be simple enough so that only the information relevant 
to the learning task was provided (Spaulding, 1956; Travers & Alvarado, 1970), the present study found that the 
addition of colors in motion visuals can help third-graders memorize and comprehend the instructional 
materials. There were two possible explanations which may illustrate this situation. First, some research 
found that since chromatic information can delineate figure/ground relationships, it appeared to aid cognitive 
tasks such as recall, free association, and recognition memory (Stone, 1983; Pruisncr. 1993). Furthermore, 
research also indicated that the younger children recalled the visual information by color more quickly than by 
shape (Pick, Christy & Frankel, 1972). 

The second likely reason was that due to the rapid advances in technology recently, visual information was 
often displayed by adding more and more details, colors etc. Therefore, children may feel uncomfortable 
watching instructional material without vivid chromatic information. As a result, visual information with only 
outline drawings may not motivate students by fourth-grade to learn the instructional contents. 
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The Effects of Visual Information on Memory and Comprehension 

The data analysis of this study shown that visual complexity in motion visuals had different effects on 
third graders’ memory and comprehension. The addition of colors appeared to aid subjects memorizing and 
comprehending the learning contents, but the additional background information contributed only to higher 
level cognitive tasks such as comprehension. 

Gorman (1973) has suggested a tolerance level hypothesis to explain the nonsignificant research results on 
the relationship between pictorial detail and grade level. That is, due to a tolerance level existing for some 
irrelevant information in a picture, the irrelevant detail may not interfere with learner's processing of the 
relevant information presented. Chen (1993) has inquired whether the tolerance level hypothesis associated 
with cognitive level of the task. The findings of this study verified this inquiry. In term of third-grade's 
memory learning, the level of visual information contained in the ODC version was just adequate, while the 
information in the ODCB was at the extreme end of third-graders’ tolerance level. Therefore, students’ 
memory performance in the ODCB was worse than in the ODC. However, in term of comprehension learning, 
third graders’ tolerance level for some irrelevant visual information was higher. Thus, the learning 
performance in the ODC and ODCB exceeded in the OD treatment. In other words, the addition of 
background information in visuals may not interfere with third-grade children’s comprehension learning. 

Why does visual complexity have different effects on memory and comprehension learning? Miller and 
Burton (1994) suggested one possible explanation that the speed and accuracy of recall was directly dependent 
upon how the information was encoded and the attention being given to the stimulus. Therefore, in term of the 
memory' level, irrelevant information in the instructional material may distract student’ attention from the 
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essential information. Such information could not promote memorizing the instructional material and may 
even interfere with it. In contrast, in term of the cognitive level of comprehension. Ellis and Hunt (1993) 
indicated that comprehension was the process of extracting the general meaning of a communication and 
distract details. Thus, the addition of visual information may not have negative influence on the process of 
comprehension. In other words, children’s tolerance level was higher in comprehending the learning tasks 
than in memorizing them. Furthermore, Dwyer (1978) has stated that “....(comprehension) did not require 
student to utilize effectively the information presented in the illustration” (p.33). However, from the results of 
this study, it showed that children’s tolerance level may be one of the possible explanation. 

Educational Implications and Recommendations 

There were several important implications and recommendations for educational researchers and 
instructional designers to consider from the results of this study. 

1 . Instructional visuals in motion visuals selected for third graders or younger should be rich, colorful 
imagery in order to attract their attention and improve their achievement. 

2. For children by six grade, visual complexity in motion visuals may not be an important factor to be 
considered, because they have acquired the ability to process visuals effectively. 

3. Visual complexity in motion visuals affect third grader’s memory and comprehension differently. 
Therefore, whether to include background information in instructional visual messages depends on the 
types of instructional objectives. 

4 . Since the subjects selected for this study were normal students, the same research could be conducted with 
special children with different areas of retardation. 
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Abstract 



The purpose of this study was to investigate the effects of computer-aided videodisc-based 
anchored instruction on promoting students' problem-solving skills for Chinese (Taiwanese) elementary 
students. Results from a two-way Repeated Measures ANOVA showed that student problem-solving 
skills were improved significantly with anchored instruction [F(J,34)=26. 76, p<.000j. In summary , the 
finding suggested that anchored instruction provided a more motivating environment where enhanced 
students' problem-solving skills. Results also indicated that all students benefited from anchored 
instruction on the effects of problem-solving despite of their mathematics and science abilities. This 
study was significant because it provided empirical evidence for its effectiveness on teaching problem- 
solving strategies for Chinese (Taiwanese) fifth-graders. 

Introduction 



The topic of situated cognition and learning has recently received a great deal of attention in the 
communities of educational researchers, particularly of educational technologists. Based upon the notion 
that knowledge is contextually situated and is fundamentally influenced by the activity, context, and 
culture in which it used, situated learning offers an approach to structuring learning experiences that 
captures both experiential and reflective dimensions of cognition and to enculturating students into 
authentic practices through activity and social interaction in a way similar to craft apprenticeship 
(Brown, Collins, & Duguid, 1989; McLellan, 1996). With the widespread application of multimedia 
technology, the notion of situated leaning can be best achieved because computer technology has been 
deployed to expand the power and flexibility of resources. There were groups of researchers who have 
been developing and studying learning environment based on the theory of situated learning to 
maximize learning in school classrooms. One successful example was anchored instruction. Anchored 
instruction, proposed by the Cognition and Technology Group at Vanderbilt University (CTGV, 1990, 
1991,1992,1993), aims to help students develop the confidence, skills, and knowledge necessary to solve 
problems and become independent thinkers. By taking advantage of the emerging of videodisc and 
multimedia computing technology, its major features are: (1) instruction is anchored in a videodisc- 
based, problem-solving environment; (2) instruction itself provides multiple perspectives; (4) instruction 
promotes everyday cognition and cognitive apprenticeship; (5) information is designed in an embedded 
format. Based on the theories of situated learning, cognitive apprenticeship and cooperative learning, 
anchored instruction made it possible to provide a life-like inquiry situation for teaching several 
mathematical concepts and problem-solving strategies (Bransford & Vye, 1989; Cobb, Yachel & Wood, 

1992) . The perfect program of anchored instruction developed by the Cognition and Technology Group 
at Vanderbilt (CTGV) was called The Ach f entures of Jasper Woodbury Mathematical Problem Solving 
Series (The Jasper Series ). The Jasper Series are based upon set of theory-based design principles, such 
as video-based format, narrative with realistic problems (rather than a lecture on video), generative 
format, embedded data design, problem complexity, pairs of related adventures, and links across the 
curriculum. These design principles have been described in detail elsewhere (e.g„ CTGV, 1991, 1992, 

1993) . The materials were developed to teach mathematics, mathematical problem solving, and critical 
thinking skills in 5th to 8th grade classrooms. At the core of each adventure is 15-25 minute video- 
based story that presents a complex problem with a mathematical solution. To solve the problem, 
students must generate the appropriate subgoals in each adventure and 10-15 steps involved in achieving 
a solution. 



Research on the Jasper Series showed many important and positive evidences for its effectiveness 
on promoting problem-solving abilities as well as enhancing attitudes toward mathematics and 
instruction (CTGV, 1990, 1991, 1992; Hickey, Pellegrino, Goldman, Vye, Moore, & CTGV, 1993 ; Van 
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Haneghan, Barron, Young, Williams. Vye, & Bransford. 1992 * Shyu,1996). There was also ample 
evidence that situations involving the use of instructional technologies that are authentic, relevant, and 
stimulating to the learners is likely to influence attitudes and thinking skills(Simonson & Maushak. 
1996). However, the contexts of the Jasper Series were anchored in American culture. Does anchored 
instruction contribute to Chinese (Taiwanese) student's problem-solving as well as promoting attitudes 
toward mathematics? How does it work? This study tried to answer the above questions. The purpose 
of this study was to design a video-based anchored instruction and to investigate the effects of computer- 
aided videodisc-based anchored instruction on promoting students' problem-solving skills for Chinese 
(Taiwanese) elementary students. 

Methods and Techniques 

Subjects 

A class of thirty-seven fifth-graders from an elementary school located in suburban area of Taipei city 
in Taiwan were randomly selected and divided into six groups according to their mathematical and science 
abilities. Two groups were high-ability groups; two were middle-ability, two were low-ability. 

Materials 

A videodisc, Encore's Vacation, was used in this study. Encore's Vacation is a techno logy -based 
program designed to motivate students and help them learn to think and reason about complex problems in 
mathematics learning. It involved the adventures of a college student named Encore with three of his 
classmates. The story begins with four college students planed to arrange a trip by train. During their trip, 
they had to change their schedule because someone had an accident. How did they make a reasonable 
decision to back home on time? Did they have to reschedule their trip? Why? How did they share the 
expenses spent on this trip? The videodisk started as a highly motivating third-person experience — 
watching a linear story told via video. But, it finally became a personal (first-person) experience when 
students actively engage in helping the actors in the story solving a meaningful, real-life challenge. An 
important design feature of the Encore's Vacation, similar to the Jasper series, was the "embedded data 
design." Student had to generate the problems to be solved and then find relevant mathematical information 
presented throughout the video story. All the data needed to solve the problem were embedded in the story. 

The sequencing of the videodisk instruction was programmed and controlled by a personal computer 
(486) and Pioneer model VD-4400 laserdisc player using the Authorware Professional for Windows 
software. 

The instruction was needed to be done through many problem-solving steps. The activities in the 
curriculum included watching the story from the videodisk, learning the strategies in solving problems, and 
solving the (mostly mathematical) problems cooperatively presented in the videodisk. 

Procedures and Treatment 

This experiment was one-group time series with pretest and posttest design. Subjects were given an 
orientation of the instruction first and then they were asked to watch the story' Encore's Vacation linearly. 
Then, subjects was given a pretest based on the content of the story. The scores of this pretest were used as a 
baseline to examine what strategies students used in problem-solving process before the treatment. The 
treatment was then followed. The treatment was the instruction. It began with the presentation of the video 
segments. At the end of the story, the video presented problems and challenged subjects to solve the 
problem cooperatively. The instruction used totally eight periods of class in a week. During the instruction, 
the role of teacher was to guide students to recognize the problems and to provide them necessary 
scaffoldings to solve problems. After the treatment, all students were administered a 12-item posttest 
(parallel to the pretest but not the same). Students in the treatment investigated two major questions. As 
each question (i.e., calculate time and money) was introduced in class, students were encouraged to generate 
subordinated questions of the stated question and to recall the facts and retrieve relevant data from the 
videodisk to answer the questions. This segment of instruction was designed to engage students in planning 
for problem solving and to focus their attention on gathering the needed information. Students were guided 
to generate complete solutions for all of the sub-problems identified. As sub-problem solutions were 
generated, students were encouraged to relate the solutions to the overall problem. Students engaged in 
problem solving in a small-group format. Each group included six persons. 

Students under the treatment were divided into groups and worked collaboratively, but they were 
given the tests (both pretest and posttest) individually. Four-step problem-solving procedure was measured 




and scored for both pretest and posttest. They were problem-identification, problem-formulation, subgoal- 
generation, and solution execution according to Polya‘s mathematical problem solving model (1957). Tests 
were scored by three experts, with an inter-ratter reliability of 0.965. The procedure of this study is shown 
on Figure 1. 

Figure 1. Flowchart of the treatment 




Research Design and Research Questions 

This study was a field-experimental, one group time series with pretest and posttest design. It aimed 
to answer the following three research questions: (1) Did anchored instruction help to promote students' 
problem-solving abilities for 5th-graders? (2) For different-ability (high-, medium-, low-ability) students, 
did anchored instruction help to promote their problem-solving abilities? (3) Were there any significant 
difference among ability groups (high, middle, low) on their improvement of problem-solving abilities? The 
independent variables in this study were group(i.e., high-, medium-, and low-ability group) and time 
series(i.e., pretest, posttest). The dependent variables were scores on problem-solving. Data were analyzed 
from a 3 x 2 Repeated Measures ANOVA through SPSS-x . 

Results 

Mean scores of pretest and posttest for different ability groups were listed on Table 1. Results from a 
two-way Repeated Measures ANOVA showed that student problem-solving skills were improved 
significantly with anchored instruction [F(l,34)=26.76, p<000](See Table 2). Furthermore, data also 
indicated anchored instruction obviously has contributed to students problem-solving for all three groups 
(high-, middle-, and low-ability group)( p< .01). 



Table L Mean scores of pretest and posttest for different ability students 





N 


N 


lliiiiiillll 






SD 


SD 


Ability 


pretest 


posttest 


pretest 


posttest 


pretest 


posttest 


high- 


12 


12 


15.76 


21.94 


11.09 


13.49 


mid- 


11 


11 


13.40 


21.59 


7.75 


13.72 


low- 


14 


14 


15.29 


25.65 


8.50 


14.95 


total 


37 


37 


14.88 


23.24 


9.02 


13.86 



♦Based on a total score of 60. 



Table 2. A two-way (Group x Time) Repeated-Measures A NO VA on problem-solving scores 



Tests of Between-Sub jects Effects 





i 


$$ 


wB&wmm 


!;■ : Fraiion: •••• 


P. prob. 


Within+Residual 


34 


8108.32 


238.48 






Group (H,MJL) 


2 


110*71 


55.36 


.23 


.794 



Tests of Within-Su bjects Effects 



Si .Soar&ss i 


liiilMii 


ss 


m 






Within+Residual 


34 


1579.85 


46.47 






Time(pre-,post-) 


1 


1243.64 


1243.64 


26.76 


.000* 


Group x Time 


2 


56.59 


28.30 


.61 


.550 


Total 


37 


3272.89 









*<.001 



This result was identical with a test of pretest and posttest across all ability groups(t=5.33; p< 000)(See 
Table 3). However, there were no significant difference among ability-groups on their increment of 
problem-solving skills [F(2,34)=.23 : p=.794](See Table 3). There was no interaction effect between three 
groups and improvement of problem-solving. In summary, the finding suggested that anchored instruction 
provided a more motivating environment where enhanced students’ problem-solving skills. Results also 
indicated that all students benefited from anchored instruction on the effects of problem-solving despite of 
their mathematical and science abilities 



Table 3. A t-test on pretest and posttest of problem-solving scores 



ft . i! Scorn 


N 


imm 


* \ 




pretest 


37 


14.88 


9.02 


t= 5.33 


posttest 


37 


23.24 


13.86 


p = .000* 



*<.001 



Figure 2 . A bar graph of pretest and posttest on problem-solving Scores 




pretest posttest 




444 



Discussion and Educational Importance of This Study 

Anchored instruction is a new instructional method that combines theories such as situated cognition, 
cognitive apprenticeship, cooperative learning and constructivist theories. With the help of interactive 
videodisk and computer technology, it presents the situations of daily life in the way of storytelling. The 
aim of anchored instruction is to provide an inquiry and real-life learning environment and authentic tasks 
that help students enhance their problem-solving skills. The results in this study indicated that students' 
using problem-solving strategies significantly differed after receiving the experimental instruction, which 
proved anchored instruction improved students’ thinking skills. Moreover, the results also showed that 
anchored instruction obviously has contributed to students problem-solving for all three groups (high-, 
middle-, and low-ability group). However, there were no significant difference among ability-groups on 
their increment of problem-solving skills. This study provided an inspiring result that all students benefited 
from anchored instruction on the effects of problem-solving despite of their mathematical and science 
abilities. 

This study supports Hickey et al.'s argument (Hickey. Pellegrino, Goldman. Vye, Moore, & CTGV. 
1993) and provides evidence that the Encore's Vacation environment is indeed highly motivating and 
enjoyable, and that the experience has a positive impact on students 4 problem solving skills. 

Numerous research (CTGV, 1991,1992; Hickey & et al„ 1993; Van Haneghan & et. al, 1992) 
provided evidence in supporting The Jasper Series (the perfect example of anchored instruction) on 
promoting problem-solving skills for American elementary students. However, none of research 
concerning the effects of anchored instruction were done for students with a different cultural 
background. This study was significant because it provided empirical evidence for its effectiveness on 
teaching problem-solving strategies for Chinese (Taiwanese) fifth-graders. 
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The Internet is changing the way researchers and practitioners develop and share information. Electronic 
publishing via the World Wide Web is generally considered a more rapid, inexpensive, and widely available 
alternative to traditional academic publishing. There are many challenges to Web-based publications, such as 
questions of copyright and quality control. Other issues go well beyond the Internet as a mere publishing medium 
such as how the Internet can be used to create a virtual community of professionals, interacting and creating 
information electronically. 

Some of these efforts have been on a relatively small scale with clearly defined goals. For example, during 
the development of the March/April, 1997 issue of Educational Technology (which focuses on Web-based learning) 
drafts of the articles were collected and posted on a Web site for review and comment by the other authors in order to 
enable cross-referencing, prevent overlap, and foster collaboration. On a much larger scale is H-NET: Humanities 
and Social Sciences On-line (http://h-net2.msu.edu), a group of listservs that tie together more than 51,000 scholars 
in 70 countries (Guernsey, 1997). Through H-NET, subscribers share ideas, solve problems, and work together on 
collaborative projects. 

A listserv is one of the most common discussion formats on the Internet. A listserv is a computer-mediated 
discussion venue whereby users can contribute information and opinions (“posts”) on a variety of topics at their 
leisure to be read by the other participants either immediately upon receipt or at a time convenient for them. Replies 
can be sent either to the list (for replies of general interest) or privately to individuals. Listservs differ from bulletin 
board services (BBSs) and newsgroups (such as those on Usenet) in that each post to a listserv is sent as an e-mail 
message to each subscriber (Ferrara, Brunner, & Whittenmore, 1991). 

A large number of listservs have been developed that are dedicated to various aspects of instructional 
technology. LM_NET is focused on school library media at the K-12 level. EDTECH is aimed at students, faculty, 
and others interested in educational technology. EDNET explores the potential of using the Internet in education. 
Other listservs focus on such topics as educational gaming, distance education, artificial intelligence, etc. The 
development of these electronic communities is changing ways in which academics develop and share information 
and raises a number of issues which need to be addressed by the academic community. This paper will discuss and 
illustrate these issues in the context of our experiences with the listserv ITForum. 



Overview of ITForum 

ITForum is an electronic listserv that discusses theories, research, new paradigms, and practices in the field 
of Instructional Technology. ITForum is affiliated by AECT’s Division of Learning and Performance Environments 
(DLPE) and the Department of Instructional Technology at The University of Georgia (UGA). The list is open to 
anyone interested in instructional technology (a special invitation is extended to graduate students in IT programs). 
The first ITForum discussion featured a paper by David Jonassen in which 84 subscribers participated. As of 
February 1, 1997, ITForum consists of a total of 722 subscribers representing 24 countries. ITForum will have 
hosted a total of 20 discussions by the end of May, 1997. 

ITForum began in May, 1994 with little worry about success or failure, largely as an experiment in 
creating a virtual community where educational technology professionals could get together to discuss important 
issues raised by leading scholars in the field. ITForum was the brainchild of Cindy Leshin, then President of the 
Association for the Development of Computer-based Instructional Systems (ADCIS). ITForum was designed to be a 
professional service to ADCIS members. Lloyd Rieber, a faculty member in the Department of Instructional 
Technology at The University of Georgia (UGA), established the list at UGA and has remained the person largely 
responsible for maintaining both the technical and professional aspects of the list. Lisa Bennett and Saada Al- 
Ghafry, two doctoral students at UGA, have worked with Lloyd as moderators and managers for the list. Recently, 
Gene Wilkinson, another UGA faculty member, has joined the collaboration, especially to help with current efforts 
to make past discussions accessible to the professional community using the World Wide Web. 

After the dissolution of ADCIS in 1994, active members petitioned AECT to form DLPE. The 
professional affiliation of ITForum was then shared by DLPE and the Department of Instructional Technology at 
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UGA. It is important to note that ITForum has never received any external funding to operate and charges no fee for 
participation. Instead, it relies completely on the volunteer efforts of the moderators, authors, and subscribers. (It is 
important to note that technical support is provided by the University of Georgia Computing and Networking 
Services. Though costs to maintain a listserv are real, these are absorbed by UGA.) 

As previously mentioned, several other listservs dedicated to instructional technology issues were already 
operating at that time, however ITForum took on a mission and operating plan unique in at least two ways. First, 
ITForum is more structured than other listservs in that specific discussions are organized throughout the year. From 
September to May several leaders in the instructional technology field are invited to submit short papers or essays 
that are posted on the list. Second, and most important, subscribers then have the opportunity to discuss the paper 
and its ideas directly with the author. Subscribers post any comments or questions they wish directly to the list. 
The author then responds to these each day for one week, either addressing specific comments and questions or 
general themes. 

Dealing with an Iterative Mission and Purpose 

There has been no attempt to establish formal policies and procedures for ITForum beyond the goal of 
soliciting papers or essays from leaders in the field and offering a discussion forum with subscribers. Papers are 
solicited solely at the discretion of Lloyd Rieber, the official owner and chief moderator of the list. Of course, he 
tries to carefully consider the recommendations of the subscribers. In a sense, ITForum has been shaped by its 
subscribers. We have likened ITForum discussions to the informal conversations that occur among colleagues after a 
particularly good keynote presentation at a conference. We have tried to make ITForum a place where people feel 
free to make casual comments about issues confronting the field. Again, one important distinction between 
ITForum and these "hallway chats" is that everyone has the chance to discuss the talk with the speaker as well as 
with colleagues from around the world. 

A variety of implicit operating procedures have developed over time. As already noted, the authors are 
selected on an invitation-only basis and they are free to choose their own topic. ITForum does not accept unsolicited 
papers nor has plans to do so. There has also been a deliberate effort to identify authors from different countries. So 
far, seven different countries have been represented by the authors of the 20 discussions held or scheduled to date. 
There has also been an effort recently to solicit participation from established scholars who are already subscribers to 
ITForum. At least half of the authors, however, were not aware of ITForum before they were invited to participate. 

The issue of formalizing ITForum's operating policies and procedures has been raised. So far, Lloyd Rieber 
has decided to keep the procedures loosely organized, so as not to fall into an established routine prematurely or 
unnecessarily. Part of the attraction of ITForum is its ability to improve continually its operating methods. We 
expect the purpose and mission of ITForum to change and grow as warranted by the membership. 



Publishing as a Process Rather than a Product 

The interactive nature of ITForum provides some unique benefits for both subscribers and authors. Unlike 
traditional journals or books, ITForum provides subscribers with direct access to authors. They can pose questions 
and make comments for the author to respond to during the discussion week. How many times have you read an 
article and wished to be able to talk with the author? Perhaps you’ve even tried to actually contact some authors 
through phone or e-mail. ITForum provides subscribers with the opportunity to engage not only in a scholar’s 
work, but with the scholar directly in a way that few other media are able. 

The number of ITForum subscribers has been increasing at a steady pace since it began, although the 
number of new subscribers closely matches the number signing off. (Like all listservs, there is a high rate of 
attrition among subscribers — people come and go frequently.) Why do people subscribe to ITForum? Although it 
may be a required activity for some graduate students, the greatest appeal simply remains the opportunity to read and 
discuss the scheduled papers. Many other reasons exist as well. For example, many subscribers report that there are 
few other educational professionals besides themselves at their workplaces. Hence, they often feel isolated from the 
profession and struggle to remain current with the latest ideas. ITForum provides many subscribers with a daily link 
to the profession. 

ITForum also offers authors with unique opportunities. The reason why authors write, of course, is to be 
read. A strong motivation for engaging in scholarly work in written form is the possibility that people will find 
one's work interesting and relevant. Most authors are grateful to hear from readers and enjoy the opportunity to 
engage in a dialogue of their work. Most authors would find much satisfaction in knowing that over 700 
professionals around the world are setting aside time in any one week to read and discuss one of their papers or 
essays. Participation in ITForum means just that. 
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Another opportunity for authors goes well beyond motivation or the personal satisfaction of knowing that 
there are people reading their finished works. Although riskier than submitting a finished paper, ITForum provides 
authors with the chance to engage readers during the writing process and before a work is completed. Unlike 
traditional print media, such as professional journals and books, authors can use ITForum to solicit reactions and 
feedback from readers. This has particular advantages if the author is trying to develop new ideas. While it is 
common for an author to get feedback from colleagues or students, ITForum gives authors a chance to informally 
discuss their ideas with people with a wide range of experiences, backgrounds, and even cultures. Of course, not all 
authors have been willing to use ITForum in this way and not all subscribers are equally open to reacting to works 
in progress. But the potential to provide a free exchange between scholars and readers while a project is in its 
formative stages is one of the most exciting prospects for the future of ITForum. 

Access to People with Different Backgrounds , Perspectives ; and Cultures 

Professional papers go through an expert review process before being published. In the past, the expert 
reviewers were mainly from the country where the books or journals were published. International reviews, if any, 
were not the norm, probably because of the time it takes for postal mail to travel between countries, problems 
involved with international coordination, and possibly the costs involved too. 

With the advent of the Internet it is possible to get international expert review. On ITForum, authors get 
instantaneous feedback from professionals of varied cultural backgrounds. Though few of the articles have dealt 
specifically with cultural matters, many argue that cognition itself is largely a cultural process. An author writes 
from his/her own perspective, mostly oblivious of how readers from other cultures will perceive the article. For 
example certain acronyms that are assumed to be understood by authors or participants from the USA might mean 
different things in other parts of the world. The advantage on ITForum is that the author gets feedback from people 
with a myriad of international perspectives. Some authors accept the input and rethink their ideas whereas others are 
less willing to accept viewpoints different from their own. Whatever the result, in one week of discussion the author 
and subscribers are "forced” to see alternative viewpoints and confront their own biases, prejudices, and 
ethnocentrism. 

Promoting this kind of reflection is difficult without the global connectivity afforded by the Internet. 
ITForum encourages authors and subscribers to think in truly "global village" terms. The ITForum schedule is no 
longer described in terms of "fall, winter, and spring," for example, since climatic seasons are not identical around 
the world (as subscribers in Australia quickly reminded the moderators early on!). The process of thinking beyond 
one’s own cultural borders might be difficult but the result can be a significant step towards developing intercultural 
understanding, empathy, and tolerance to views that differ from one's own. 

Developing a Research Community 

Through ITForum, subscribers become members of a larger academic community that is spread over the 
world. It doesn’t matter whether they are active participants in the discussions or passive "lurkers" who silently 
follow the thread of the conversation. Even without meeting face-to-face, friendships and active collaborations often 
develop between subscribers through the listserv. The informal exchange of opinions, experiences, and arguments 
during the discussions provides a much more personal element to academic discussions than can be derived from 
more formal forms of publishing. The spontaneous give-and-take of electronic communications exposes the 
character, the strengths, and the weaknesses of the participants and their arguments and allows the development of 
mental and emotional ties without the need for physical proximity. 

The existence of an electronic community can be demonstrated through the establishment of accepted rules 
of behavior (i.e. "netiquette") and through the development of shared values and experiences (Rheingold, 1995). 
Many examples of collaborative research and development projects or jointly authored scholarly publications that 
grew out of initial contacts over ITForum can be cited. The development of a community is probably best 
illustrated, however, by a few personal examples. Lloyd Rieber commented that when he visited Edith Cowan 
University and Curtin University of Technology in Perth, Australia he felt as though he was renewing friendships 
that had already been established through ITForum even though he had never actually met many of the people 
beforehand. The recent death of longtime ITForum participant Jeff Oliver of the United Kingdom was felt as a 
personal loss by many of us who had never met him face-to-face but felt that we "knew" him through his insightful 
posting to the list. 



Potentials and Pitfalls of Computer Mediated Communication (CMC) 



0 

ERIC 



Although the various forms of CMC differ, the asynchronous types of CMC (personal e-mail, listservs, 
bulletin boards, newsgroups) have some important characteristics in common. Each message is rendered in visual 
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text only without the usual conversational cues of body language, changes in vocal tone or pitch, or other 
paralinguistic clues. This lack of cues can lead quite easily to misunderstandings between users. For example, 
sarcasm is not conveyed well via e-mail (Newby, 1993). Users of CMC, however, have developed some norms 
unique to CMC in order minimize misunderstandings. These include "emoticons" (emotion icons) such as the 
standard "smiley" typed as :-) that resembles a smile if you tilt your head to the left. The smiley is usually added 
after a line to indicate “I’m only kidding, don’t take what I’m saying too seriously" (Murray, 1988). 

Besides removing the barrier of time, CMC also removes the barrier of distance, allowing conversations 
between people separated physically and/or temporally. On ITForum, this has allowed scholars and IT professionals 
from all over the world to participate in discussions without consideration of geography or time zones. 

Netiquette: Reliance on Self- Moderation 

Since the inception of e-mail, users have been struggling to develop a set of standards of use, known as 
"netiquette." Some of the ‘‘rules’’ are to post only replies of general interest to the list (items of personal interest 
should be sent off-list) and to avoid sending any e-mail when angry or upset. Some list owners have had trouble 
getting their members to abide by these rules and have therefore begun “moderating" the list discussions. In a 
formally moderated list all posts must first be sent to the moderator who clears the ones deemed appropriate and 
posts them to the list. Personal, off-topic, or inflammatory posts are rejected. Since ITForum allows subscribers to 
post notes directly to the list without first getting them "cleared" by a moderator, it relies heavily on subscribers 
carefully considering what they decide to send. One of the most remarkable things of ITForum is the natural 
tendency of the list to moderate itself. With few exceptions, ITForum has largely been able to avoid problems of 
other lists, such as flaming and junk mail. 

Achieving Critical Mass : Avoiding the Negative Side of Growth 

ITForum exists by the very fact that people are willing to participate by sending their comments to the list. 
Participation defines the very nature of ITForum. On one hand, the number of subscribers (and the rate of 
increase/decrease in subscriptions) can be perceived as a measure of the value of ITForum to the profession. On the 
other hand, growth also holds several potential problems for the future of a listserv like ITForum. The first simply 
relates to the volume of mail that would be generated if even a small percentage of the subscribers were to post 
something to the list every day. Our subscribers are busy professionals who do not have the time to sort through a 
large number of messages. The second problem concerns the ratio of "fluff to stuff' in the daily ITForum postings. 
As the subscriber pool increases, there is a greater tendency for people to post notes or questions that have little 
relevancy to majority of the subscribers. 

In addressing the second problem, we have asked subscribers to keep all postings during scheduled 
discussion weeks directed toward the topic and to hold off sending other postings until the discussion has ended. 
Problems still arise when too many notes are received, even if they all are relevant to the topic. As already noted, 
we have tried to invite casual comments from subscribers, yet if more subscribers accepted our invitation the number 
of messages would be overwhelming. This dilemma does not have an easy resolution. Again, we rely on the 
membership to self-moderate itself. So far, we have been fortunate that membership interests and discussion topics 
vary widely, hence just a small proportion of the subscribers contribute to the discussion on any given topic. 

When Viewpoints Conflict: Dancing on the Lip of the Volcano 

ITForum offers subscribers an uncensored exchange of ideas and perspectives. Not surprisingly, viewpoints 
among subscribers often conflict. Fortunately, this usually results in our best discussions. ITForum subscribers 
have shown their willingness to listen and respect all viewpoints. Occasionally, however, emotions begin to take 
control of a discussion. It is interesting to watch how a discussion can quickly grow more intense within a short 
period of time. The immediacy of e-mail permits people to post and respond at will. The ease of pressing the 
"send" button sometimes leads to remarks that would probably not have been made in another medium. Although 
we feel that subscribers have rarely, if ever, crossed the proverbial "line in the sand" into unprofessional behavior, 
such dangers remain a constant presence in a medium such as this. 

Providing Access to Past Discussions: Can You Catch the Wind ? 

The papers presented over ITForum are a rich resource of ideas related to many of the major issues of 
concern to students, practitioners, researchers, and scholars in the Field of instructional technology. Questions related 
to developmental strategies and processes, research methodology, questions, and ethics, theories of knowledge 
development and techniques of enhancing student learning have all be explored in recent discussions. Many of the 
papers presented have made their way into more traditional forms of publishing and efforts have been made to provide 
access to the papers on the Internet by means of the DLPE homepage (http://dlpe.base.org) and InTRO (Instructional 
Technology Research Online) (http://intro.base.org). In many cases, however, the paper has been primarily a point 



of departure and the value of the forum has been in the discussion that grew out of the paper. Capturing and 
presenting the ebb and flow of an ongoing discussion is like trying to take a snapshot of the wind. 

Like almost all listservs, past ITForum discussions are preserved for a limited time and can be accessed 
through the listserv archives. The process of locating and downloading the appropriate files, however, is confusing 
and cumbersome. Once the file has been located, the individual postings which develop the thread of the argument 
are often lost in a tapestry of competing threads. In order to provide easier access to past discussions, an ITForum 
World-Wide Web site is being established at the University of Georgia. The site is still under development, but you 
are invited to visit and provide us with comments or suggestions (http://itechl.coe.uga.edu/ITForum/home.html). 

The discussions are presented virtually verbatim as they occurred, although minor editing has been done for 
stylistic and grammatical consistency and to improve the flow of the discussion. Messages which consisted 
primarily of "cheering" (voting for or against postings by other contributors) have been eliminated as have messages 
which are clearly off the thread of the discussion. Only those messages which contributed to the development of 
issues raised in the papers have been kept. In addition, some minor rewording of some posts was done to clarify the 
intent of messages but efforts have been made to preserve the feel of a give-and-take discussion. 

Copyright: Who Owns the Discussion ? 

Making the ITForum discussions easily and widely available through the Internet raises a number of 
questions regarding such issues as rights of privacy, intellectual property rights, ethical treatment of participants, and 
copyright protection (Schrum, 1995). Some of these problem areas are relatively easy to handle. For example, 
messages are often sent in an e-mail discussion that contain grammatical or spelling errors that would not normally 
be committed by the author in a more formal communication. In such circumstances, we view our function much 
like that of an editor or proofreader in a traditional publishing house, making those corrections which should be 
made, while preserving the intent of the author. 

On questions of copyright and intellectual property rights, however, we take a fairly restricted view. 
Following the lead of H-NET (Lawrence, 1995), we feel that although posting a message to ITForum gives 
ITForum the right to publish the message over the Internet, the ideas and words of the message remain the 
intellectual property of the subscriber and that publication on ITForum does not place the material in the public 
domain. Use or publication of such material by other writers requires citation of the source and, in the case of 
publication in any other arena than ITForum, permission of the writer. When messages in the discussions quote 
previous postings, common practice within the ITForum community has been for the source of the quote to be cited. 
On the web site, we take this a step further -- the wording of all quoted messages is verified and the date and author 
of the original message are cited. The only exceptions to this are those few cases where questions of privacy rights 
come into conflict with intellectual property rights. At times personal revelations are made in the heat of a listserv 
discussion that advance the discussion but that, given time to reflect, the author might feel reveal more than they 
might wish to keep available in a more permanent and public form. In such cases, the author is consulted and the 
name deleted from the message, unless the editor is otherwise instructed. 



Summary 

ITForum is a unique and thriving community of instructional technology scholars and professionals. By 
offering a schedule of papers and the opportunity to discuss them with the author, ITForum makes a valuable 
contribution to the professional community. Students, faculty members, and IT professionals alike have an equal 
opportunity to ask questions and make observations. The authors of the papers receive immediate feedback from a 
large pool of interested and knowledgeable professionals from around the world. Consequently, listservs like 
ITForum offer the opportunity for scholarly work to become a much more interactive process between authors and 
readers than is currently the case in traditional publications. Furthermore, listservs like ITForum truly bring the 
professional community into a global perspective. Colleagues from different countries can share ideas and 
experiences as if they worked on the same floor. Unanswered questions remain, such as those surrounding copyright. 
These will take time to resolve as the medium matures. 
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Abstract 

Transfer of learning is one of the major purpose of education. Theories and research have tried hard to 
answer questions like: How is transfer occurred? How is transfer enhanced? Situated cognitive theory and 
research about Anchored Instruction together bring some positive findings. Anchored Instruction provides 
learner a situated, authentic, and social learning environment, and students learn to solve problems instead of 
facts and principles. Although students are able to solve various problems in this rich context, whether they 
could solve problems successfully in novel context remain further studied. Would it be like procedural or skill 
training that is successful in simulated training context? Trainees usually could solve similar problems in real 
context that training context simulated (near transfer), but not on far transfer problems. It is suggested that 
“knowledge abstraction” is the key process for transfer to occur, and how is knowledge abstracted from context 
affects transfer. 

This study is to investigate how different knowledge abstraction strategies affect students 4 transfer 
ability. The teach of problem-solving strategies in Anchored Instruction is considered as helping students 
abstract knowledge from context and is hypothesized thus enable them to transfer. Furthermore, will it be better 
to provide aids ( self-reflection activity ) for students to abstract knowledge by themselves? An experiment was 
conducted to compare how Anchored Instruction with (1). Teaching problem-solving strategies, (2). Practice 
various problems with self-reflection activity, (3). No knowledge abstraction activity, effect near and far transfer. 
The result indicated that all the three groups performed significantly better on solving near-transfer problems 
than on far-transfer. The self-reflection group outperformed the other two groups on far transfer. The control 
group did best on near- but worst on far-transfer. However, the group differences were not statistically 
significant. 

Background 

Transfer of learning is one of the major purpose of education. Living in this rapid-changing 
information society, ability of problem-solving and learning transfer is essential. Theories and research 
(Anderson, 1987; Cormier & Hagman, 1987) have tried hard to answer questions like: “How is learning 

transfer occurred?” and “How to enhance learning transfer?”. Theory of Situated Cognition (Brown, Collins, & 
Duguid, 1989) and studies of Anchored Instruction (CTGV, 1990, 1992, 1993; Moore, et al., 1994) together 
bring some positive answers. Aiming to avoid inert knowledge, Anchored Instruction provides a situated, 
authentic, and social learning environment in which students learn together to solve problems. Previous studies 
of Anchored Instruction suggest positive findings on problem-solving and learning transfer (CTGV, 1992; 
Greeno, Smith, & Moore, 1994; Lin, 1995). However, there are two main questions about learning transfer 
remained further research. First, what is in Anchored Instruction that affects learning transfer? Second, how 
to enhance learning transfer? 

In procedural or skill training, transfer is enhanced by using simulation that is similar to the real context. 
Trainees usually succeeded on near transfer but not on far transfer (Baldwin & Ford, 1988; Kamour, Kamouri, & 
Smith, 1986; Phye, 1989; Shih & Alessi, 1993). Near or far transfer refers to how similar the transfer context 
(setting and problem) is to the learning context. Near transfer usually refers to solving similar problems or 
performing learned skills in real setting that is also similar to the learning setting. Comparatively speaking, far 
transfer refers to solving different type of problems in settings either similar or novel to the learning setting. 
Theory and research of learning transfer focused on factors that affect learning transfer. These factors include: 
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similarities between learning and transfer contexts, use of variety of examples and practices, use of analogy, 
learners’ prior knowledge and ability..., etc. It is agreed that there is no simple relationship between similarity 
of contexts and learning transfer. Furthermore, research also suggests that knowledge need to be 
“decontextualized” to enhance further transfer (Baldwin & Ford, 1988; Beard, 1993; Bernardo & morris, 1994; 
Cormier & Hagman, 1987; Grandgeneet & Thompson, 1991; Gentner & Stevens, 1983; Lin, 1995; Paas & 
Merrienboer, 1994; Royer, 1979; Phye, 1989; Shih & Alessi, 1993). It is argued that the critical factor for 
transfer to occur is learners’ ability to abstract knowledge from the learning context. That is, to decontextualize 
knowledge from the learning context. To enhance learning transfer, especially for far transfer to occur, some 
process needs to happen. This process is what we call “knowledge abstraction." The notion of “knowledge 
abstraction” is analogical to “generalization” of cognitive skills (Anderson, 1987) or formation of “abstract 
schema” (Gick, & Holyoak, 1983, 1987). In short, we suggest that (1) “knowledge abstraction” is the key 
process for transfer to occur, and (2) how is knowledge abstracted from context would affect transfer. 

Trying to help students in Taiwan to gain better problem-solving abilities, Shyu (1995) developed an 
interactive videodisc “Anchor’s Holiday” for fifth graders to learn problem-solving on mathematics and found it 
enhanced problem-solving. Nevertheless, how “Anchor’s holiday” help learning transfer remains further study. 
During previous studies (Shyu, 1995), students first watched video then tried to solve problems in the scenario 
with team members. After finished solving the given problem, the teacher discussed solutions with students 
and taught them the general problem-solving strategies that are “abstract” (not specific to any context). This 
teaching of the general problem-solving strategies is considered a way to help students to abstract knowledge 
from context, and is hypothesized thus enable transfer. Furthermore, is there a better way to help students to 
abstract knowledge from context by themselves? Practice solving different type of problems with self- 
reflection method is proposed. The self-reflection method is to write down or draw out on paper the key words, 
questions in mind, steps to take..., etc. during problem-solving. This serves as a reflection tool of mind, thus 
called “self-reflection." It is to make thoughts, ideas, plans, questions in mind concrete and thus clearer to 
continue and to discuss with partners. The function of self-reflection is analogical to flow-charting for 
programming; flow-charting provides a reflection of the program flow and algorithm and thus enable 
programmer to dry-run and complete or debug the program. It is hypothesized that students abstract knowledge 
by themselves with practicing different type of problems with self-reflection may affect their transfer ability. 

The purpose of this study is to investigate the effects of “knowledge abstraction” on near transfer and 
far transfer. Near transfer refer to solving different problems in similar context (i.e., in Anchor’s holiday 
scenario). In contrast, far transfer refers to solving different problems in a novel context (i.e., a written 
scenario that is irrelevant to Anchor’s holiday). An experiment was conducted to compare how Anchored 
Instruction with (1). Teaching problem-solving strategies, (2). Practice solving different problems with self- 
reflection activity, (3). No knowledge abstraction activity, affect near and far transfer. 

Research Questions 

1 .Does “knowledge abstraction” affect learning transfer? 

2. Do different knowledge abstraction activities (teaching problem-solving strategy; practice solving 
different problems with self-reflection; no knowledge abstraction activity) result different effects on 
near and far transfer? 

3 . Do different knowledge abstraction activities affect near transfer? 

4 . Do different knowledge abstraction activities affect far transfer? 

Method 

A three (Knowledge Abstraction Activity: teaching problem-solving strategy; practice solving different 
problems with self-reflection; no knowledge abstraction activity) by two (Learning Transfer Level: near transfer; 
far transfer) split-plot design of experiment was conducted to answer research questions (Table 1). 

Seventy-two fifth graders participated the study. Students worked together in teams. A team 
consisted of three students who have high, middle and low ability on math according to their previous math 
scores. The team members were randomly chosen from the three groups of high, middle, and low math ability. 
Twenty- four teams were randomly assigned to one of the three experiment groups (Group D, Group S, and 
Group C). 
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Table 1. Design of experiment 



Knowledge Abstraction Activity 


Learning Transfer Level 


Near Transfer 


Far Transfer 


Group D 

teaching problem-solving strategy 


NT1 


FTl 


Group S 

practice solving different problems with self- 
reflection 


NT2 


FT2 


Group C 

no knowledge abstraction activity 


NT3 


FT3 



The procedure of the experiment is as below: 

1 . Formed teams and assigned them into treatment groups. 

2 . Each team in each group used “Anchor’s Holiday” IVD, watched video and then solved problem A. 

3 . For Group D. 

Each team watched solution of problem A in “Anchor’s Holiday," then teacher discussed solution with 

students and taught problem-solving strategy. 

For Group S. 

Each team watched solution of problem A in “Anchor’s Holiday," then teacher gave problem B and 

directed them to solve the problem with self-reflection method. 

For Group C. 

Each team watched solution of problem A in “Anchor’s Holiday." 

4 . All the student took near-transfer test. 

5 . All the student took far-transfer test. 

The whole procedure took seven hours to complete. It was divided into sessions on different days in 
one week. Test scores of near transfer (NT) and far transfer (FT) were analyzed with method of 3x2 
MANOVA repeated measures. Students’ solution of problem A, records of self-reflection on solving problem 
B, observation records during experiment, and students’ previous math scores were also collected to help 
interpreting the results. 

Results 

Measures are post-test scores of near and far transfer tests: NT and FT respectively. The near transfer 
test asked students to solve a problem that has the same scenario to “Anchors Holiday." However, the 
problem in test is about budgeting, which is different from the practiced scheduling problem. The far transfer 
test also asked students to solve a budget problem, but in a completely different scenario. That is, the far 
transfer problem is different from the learning one in terms of both scenarios and type of problem. Students 
were asked to write out every step they took to solve the problems. Scoring method was borrowed from 
protocol analysis (Williams, Hollan, & Stevens, 1983). The researchers and teachers first enumerated all the 
solution steps (mental process) needed to solve the problems. It took 32 steps to solve the near transfer 
problem, and 50 to solve the far transfer problem. These steps were used as standards to score the near and far 
transfer tests. 

Two graders compared students' solution steps to the “standard steps” respectively. Each 
corresponding step scored one point. The inter-grader reliability was high, r=0.95 for near transfer and 0.91 for 
far transfer. The points students received from the two graders were averaged and then calculated to get 
percentage scores. For example, if a student received 25 points (averaged score from two graders) for near 




transfer, then his score (NT) is 78% (25/32*100). The far transfer score (FT) was obtained the same way. 
Table 1 summarizes the means and standard deviations of the near and far transfer scores (NT and FT) of the 
three groups. 







NT (Near Transfer) 


FT (Far Transfer) 


Learning 


Transfer 


Group 


N 


M 


SD 


M 


SD 


M 


SD 


Group D 


24 


43.55% 


14.73 


63.75% 


24.02 


53.65% 


14.76 


Group S 


24 


47.20% 


26.49 


67.92% 


26.42 


57.56% 


22.68 


Group C 


24 


50.91% 


22.83 


63.04% 


26.01 


56.98% 


22.25 


Overall 


72 


47.22% 


21.81 


64.90% 


25.24 


56.06% 


20.02 



To analyze data, we employed a 3 (Group: between-subjects) by 2 (Learning transfer: within-subjects) 
MANOVA repeated measure as statistical analysis method. The three groups: teaching (Group D), self- 
reflection (Group S), and the control group (Group C), belonged to the between-subject factor (Group). NT 
and FT are dependent variables that belonged to the within-subject factor (Learning Transfer). Table 2 reports 
summary of the analysis. 

Table 2. Summary Table for the 3 (Group: teaching, self-reflection, control) by 2 (Learning Transfer: near , 



far) MANOVA 



Source 


ss 


DF 


MS 


F 


Between 

Group 


426.37 


2 


213.18 


0.26 


Error (b) 


56465.03 


69 


818.33 




Within 

Learning Transfer 


11253.67 


1 


11253.67 


36.03” 


Group x Learning Transfer 


556.14 


2 


278.07 


0.89 


Error fw) 


21552.27 


69 


312.35 
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Question 1 Does “knowledge abstraction” affect learning transfer? 

The effects of knowledge abstraction on learning transfer were examined by comparing the learning 
transfer scores among the three groups, i.e. the main effect (Group) of the MANOVA analysis. Results 
indicated that the difference was not significant (F 2 , 6 9 = -26, p >.05). Figure 1 shows the averaged learning 
transfer scores (near and far transfer) for the three groups. Although the three groups did not show significant 
difference on solving transfer problems, Group S (self-reflection) outperformed the other two groups while 
Group D (teaching) was inferior to the other two groups (53.65%, 57.56%, and 56.98% for Group D, S, and C 
respectively). 

58% 

56% 

54% 

52% 

50% 

Group Group S Group 
D C 

Figure 1. Learning Transfer Scores for the three Groups 
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Question 2 Do different knowledge abstraction activities (teaching problem-solving strategy; practice 
solving different problems with self-reflection; no knowledge abstraction activity) result different 
effects on near and far transfer? 
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This question could be answered by examining the Group by Learning Transfer interaction of the 
MANOVA analysis. Although there exists significant difference between near and far transfer (F 1 69 =36.03, 
p<.01**) for the three groups, results did not show significant interaction (F 2 . 6 9-0.89, p>.05). The different 
knowledge abstraction activities did not result significant different effects on near and far learning transfer. 
Figure 2 shows the near transfer score (NT) and far transfer score (FT) for the three groups. Group C, the 
control group, did best on solving near transfer problem (43.55%, 47/20%, 50.91% for Group D, S, C 
respectively), but did worst on solving far transfer problem (63.75%, 67.92%, 63.04% for Group D, S, & C). 
Group D (teaching) did worst on solving near transfer problem and a little better than Group C on far transfer. 
Group S (self-reflection) was in-between the other two groups on solving near transfer problem, but 
outperformed the other two on far transfer. 




- - O - - Group D 
— w — Group S 

— -A — Group C 



Figure 2. NT and FT for the three Groups 



Question 3 Do different knowledge abstraction activities affect near transfer? 

The comparisons on near transfer scores (NT) among the three groups could answer this question. 
The simple effect examination of NT from the 3 by 2 MANOVA showed no significant difference among groups 
(F 2 .i38-.57, p>.05). That is to say, different knowledge abstraction activities did not result significant 
difference on solving near transfer problem. Figure 3 shows the near transfer scores (NT) for the three groups. 
The control group (Group C), although did not have any knowledge abstraction activity during learning, 
outperformed the other two groups (43.55%, 47.20%, 50.91% for Group D, S, & C). Group D, received 
teaching of general problem solving strategy from teacher, did worst on solving near transfer problem. Group 
S, used self-reflection strategy to abstract knowledge during learning, performed worse than Group C but better 
than Group D on solving near transfer problem. 




Figure 3. Near Transfer Scores (NT) for the three Groups 



Question 4 Do different knowledge abstraction activities affect far transfer? 

The comparisons on far transfer scores (NT) among the three groups could answer this question. The 
simple effect examination of FT from the 3 by 2 MANOVA showed no significant difference among groups 
(F 2 . 138 -.29, p>.05). That is to say, different knowledge abstraction activities did not result sognofocant 
difference on solving far transfer problem. Figure 4 shows the far transfer scores (FT) for the three groups. 
The control group (Group C), although did best on solving near transfer problem, did worst on solving far 
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transfer problem (63.75%, 67.92%, 63.04% for Group D, S, & C). Group D who did worst on near transfer test, 
performed a little better than Group C on far transfer test. Group S outperformed the other two groups on 
solving far transfer problem. 




Figure 4. Far Transfer Scores (FT) for the three Groups 

Discussion 

Results of this study indicated that different knowledge abstraction strategies (teaching general 
problem-solving strategy, practice with self-reflection strategy, or no strategy) did not significantly affect 
learning transfer. However, results showed that each group performed significantly better on solving far 
transfer problem than near transfer problem. Nevertheless, different knowledge abstraction activities did not 
result significant difference on solving different level of transfer problems, nor on near or far transfer 
respectively. Although the results did not show any significant effect of knowledge abstraction strategies, there 
are some interesting points to discuss. 

First, all students performed better on far transfer test than near transfer test. This result is quite 
abnormal to previous findings (Baldwin & Gord, 1988; Cormier & Hagman, 1987; Royer, 1979; Shih & Alessi, 
1993). Usually, far transfer task is more difficult than near transfer task for students to perform, and students 
usually scored higher on near transfer test. One possible reason is that the near and far transfer tests have 
something in common (both asked students to solve budgeting problems). Therefore, students also “learned” 
on taking near transfer test and thus did better on the following far transfer test. Also, the test format is also a 
possible reason. On taking near transfer test, students were for the first time trying to write the test in the 
format. It probably took some efforts to figure out how to write the test. While on taking the far transfer test, 
although the problem was different, the test and the way to answer it were same as taking the near transfer test. 
Students could concentrate more on solving the problem and might thus do better than near transfer test. 

Second, Group C did best on near transfer test and worst on far transfer test. Group C did not have 
any knowledge abstraction strategy during learning. That means, they learnt from the Interactive Video on their 
own. Since the near transfer test was budgeting questions in the same scenario as in IVD, they could use 
whatever solutions they obtained during learning to solve the similar problem. This might be easier to do while 
the other two groups were involved in “knowledge abstraction:" However, when solving the far transfer 
problem, Group C might have not abstracted knowledge from previous learning and found it difficult to solve the 
far transfer problem. Therefore, they did worse than the other two groups (Figure 2). That means, the other 
two groups spent some effort on “knowledge abstraction” and resulted some effects on transfer, although not 
significantly. 

Third, Group D performed worse than the other two groups in terms of learning transfer (Figure 1 and 
Figure 3). This result is a little surprising. Group D received teaching of general problem-solving strategy 
from teacher after learning from IVD. Previous studies of Anchored Instruction used this method and found 
positive results on students 4 problem-solving ability. However, in this study, compared to the other two groups, 
received teaching of problem-solving strategy seemed increased difficulties on solving transfer problems. 
Probably students had difficulties on transferring the taught problem-solving strategy to solve different 
problems. 

Forth, the self-reflection group (Group C) increased more on far transfer scores than the other two 
groups (Figure 2) and did best on the far transfer test (Figure 4). This result is as predicted. Students learned 
from IVD by using self-reflection strategy to discuss and solve problems. Presumably, self-reflection strategy 
allowed students to write or draw out what they thought in mind and thus enable them to clearly identify key 
points, enable better communications and discussions, lay out the possible solutions and “dry-run” it, thus were 
more able to abstract knowledge from the context. Therefore, Group S increased their transfer ability more. 




Although Group S was inferior to Group C on near transfer (Figure 3), but in far transfer and overall transfer 
tests, Group S outperformed the other two groups (Figure 1 & 4). 

In summary, the effects of knowledge abstraction strategies on learning transfer were not statistically 
significant in this study. However, the results showed some effects and suggest further study on knowledge 
abstraction. The main possible reason that the effects were not obvious is that the experimental time (one week) 
was too short. It probably took longer for students to develop abstract knowledge, whether being taught or 
using self-reflection method. Furthermore, another interesting question would be how people abstract 
knowledge? Would different knowledge abstraction strategies endow different abstract knowledge? What is 
the knowledge abstraction process? These questions would require qualitative research method to answer. 
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Abstract 

An experiment is reported in which subjects work with an interactive videodisk program about cheese making in 
different experimental conditions with respect to learner control and pre-instructional strategies. The purpose of the 
study is to investigate the influence of pre-instructional strategies on the relationship between learner -controlled or 
program-controlled length of video segments and on related test performance on post-tests and retention tests. Data 
were collected in three rounds in April-May and December 1996 , and in January 1997. The results show that 
program-control leads to better post test performance , although the effect size is rather small. Interaction with pre- 
instruction was not observed. 

Pre-instruction and learning from interactive video programs 

The effects of pre-instruction on learning (Hartley & Davies, 1976; Hannafin & Hughes, 1986), as well as 
the effects of interactive video on learning (Verhagen, 1992; Tovar & Coldevin, 1992; Schaffer & Hannafin, 1986; 
Phillips, Hannafin & Tripp, 1988), have been studied by several researchers. This study is an attempt to offer a 
contribution to the knowledge about the combination: the use of pre-instruction before working through an 
interactive video program. 

The literature shows a variety of definitions and forms of pre-instruction (Hartley & Davies, 1976; Hannafin 
& Hughes, 1986). A general definition, however, is not available. In this paper, pre-instruction is conceived to 
offer a framework for the actual instruction. Hartley and Davies (1976) distinguish four generally accepted forms of 
pre-instruction: pre-tests, behavioural objectives, overviews, and advance organisers. Research shows that each of 
these forms is more or less suited for reaching certain learning results (Arkes, Schumacher, Gardner, 1976; Hartley & 
Davies, 1976). 

Pre-instructional strategies, for instance, appear to be useful to support meaningful learning. The effect of 
different forms of pre- instruction appears to be dependent on the ability level of the learner. Learners with relatively 
low cognitive abilities benefit fairly significantly from overviews, for middle ability learners behavioural objectives 
are more suited, and high ability learners benefit more than the others from pre-tests, overviews, and advance 
organisers (Hartley & Davies, 1976). Derry (1984) concluded that establishing links with existing cognitive 
structures before instruction yielded positive effects on long-term memory of elements from a text. Research shows 
that orienting objectives often improve the learning process. One theory about the influence of learning objectives 
on the learning process is that the objectives direct the attention of the learner to relevant aspects of the subject 
matter (Hartley & Davies, 1976). Kaplan and Simmons (1974) concluded that a consistent relationship between 
learning objectives and subject matter is a necessary condition for the effect of learning objectives on learning. This 
implies that the objectives should cover the content to be learned. They also concluded that the use of learning 
objectives before the instruction leads to less incidental learning. Incidental learning is described as learning that 
occurs while studying subject matter to reach certain learning objectives but which is not relevant for these 
objectives. Intentional learning, on the contrary, is directly relevant for the related objectives. According to Gagne 
(1985), learning objectives help to create a mental set that directs the attention of the learner to important objectives 
and causes selective perception of the learning content (Klein & Pridemore, 1994). 

Other researchers (Duchastel & Brown, 1974) suggest that learning objectives increase the learning of 
objectives-related content, but that it decreases the learning of remaining subject matter (Ho, Savenye, & Haas, 
1986). Learning objectives seem not to be effective in all situations. 

Research shows that presenting learning objectives facilitates the learning of factual information, but does 
not support learners to acquire higher cognitive skills (Hannafin, 1985; Ho, Savenye, & Haas, 1986; Mayer, 1984). 

The effects of pre-instruction cannot always be clearly determined. The effects may often be overruled by 
powerful factors of the instruction (Mayer, 1979; Hannafin, Phillips, Rieber, Garhart, 1988). In general, the power 
of pre-instruction diminishes as more prior knowledge is brought to the learning situation and more structure is built 
into the instruction. In spite of this problem, clear indications exist that the application of pre-instruction may have 
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a positive influence on learning results in certain situations. 

Existing studies about the effects of interactive video and pre-instruction on learning, mainly focus on the 
variable 'Locus of Control*. The results appear to be inconclusive. Hannafin and Colamaio (1987), for instance, 
conclude that designer-controlled (the designer determines the way of sequencing and presenting of information to the 
learner) and learner-controlled (the learners decide for themselves which route to take through the information) 
conditions yield significantly better test scores than a linear condition (the information is presented in a fixed linear 
sequence). Tovar and Coldevin (1992), however, conclude the opposite: linear and 'mixed control' conditions scored 
better. In a study by Breman (Breman, 1995; Verhagen and Breman, 1995), the linear condition seems to have a 
positive effect on learning results that are related to the learning of factual information. An explanation could be that 
in the linear condition, the learners were aware of the fact that they had to watch long video segments followed by 
answering questions that were related to previously presented learning objectives. The awareness of the long video 
sequences may have stimulated the learners to invest relatively much mental effort to be able to retain the presented 
information. 

The present study aims primarily at the effect of presenting learning objectives in advance on the learning of 
factual information from interactive video programs. Earlier research showed that this form of pre-instruction seems 
to have a positive effect on learning results (Tovar and Coldevin, 1992). This research looks at immediate recall as 
well as at recall over a long term. Further, the question is studied whether presenting learning objectives results in 
decreased incidental learning. Also 'locus of control* receives attention. Two versions of the same interactive video 
program, a linear version under computer control ('FIXED') and a free version under learner control ('FREE'), arc used 
to gather more information about the influence of ’locus of control* on the learning of factual information. The 
experimental video disk program that is used for this study is the same that was used by Breman (1995) and 
Verhagen (1992). 

The central research question is as follows: 

"What is the effect of presenting pre-instruction in the form of learning objectives on learning factual 
information from an interactive video program?" 

The answer to this question is obtained by testing of the following hypotheses: 

HI Subjects who receive pre-instruction have better scores on a post test than subjects who do not receive pre- 
instruction. 

H2 Subjects who receive pre-instruction have better scores on a retention test than subjects who do not receive 
pre-instruction. 

H3 There is an interaction effect between 'presenting learning objectives’ and 'locus of control (Tovar and 
Coldevin, 1992). 

H4 Subjects in the FIXED condition score about as high or higher on a post test than subjects in the FREE 
condition (Verhagen, 1992; Breman, 1995). 

H5 Subjects in the FIXED condition score higher on a retention test than subjects in the FREE condition 
(Verhagen, 1992; Breman, 1995). 

H6 In the FREE condition, subjects who receive pre-instruction will, in general, choose shorter segment 
lengths than subjects who do not receive pre-instruction. 

H7 Subjects who work through the experimental program in the pre-instruction condition show more 
intentional learning than incidental learning. 

H8 Subjects who work through the experimental program in the no-pre-instruction condition perform worse on 
test items that relate to intentional learning than the pre-instruction group and perform better on test items 
that relate to incidental learning than the pre-instruction group. 

For H6, segment length is not measured in time (as the number of seconds that a video segment runs), but is 
operationalised as the number of information elements that are contained in the segment. An information element is 
thereby defined as the smallest unit of meaningful information in the programme, which generally takes the form of 
one sentence spoken by the narrator with accompanying video images. The experimental section of the video disk 
program contains 216 information elements, distributed over 7 chapters. The mean segment length of each chapter 
is thus about 31 information elements. Compared to measuring segment length in time, in this case one 
information element has on the average a duration of about 8 seconds, yielding mean chapter lengths of about 4 
minutes. 
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Method 



Subjects 

49 first-year students from two Agricultural Colleges in Velp and Leeuwarden in the Netherlands were the 
subjects. Their ages range from 18 to 22 years old. All students were prepared for college-level education by 
completing appropriate general or vocational programs in secondary school. 

To determine the desired sample size, the guidelines by Neter, Wasserman and Kutner (1985) were used. To 
be able to test the hypotheses in this study at a level of significance with a=0.05 and a power of 1-0=0.80, at least 
17 subjects should be selected for each factor and thus a minimum of 9 subjects per condition. This is based on a 
minimum difference A of 4 points between the means of the scores on the post test and the retention test. The 
maximum score of each test is 34 points. Further, the standard deviations of the scores on the post test and the 
retention test that were found by Breman (1995) were taken as a starting point. In April 1996, 40 students from the 
school in Velp put their names on a list to volunteer for the experiment. Participation in the experiment was an 
activity which was recognised as part of a project they were working on. Unfortunately, this activity was not 
strictly required and many students decided to spend their time on a different activity. In the end, only 24 subjects 
remained. To compensate for the loss of subjects, the experiment has been repeated in December 1996 in Velp, 
yielding 13 extra subjects, and in January 1997 in Leeuwarden, where 11 extra subjects participated. This time, the 
Velp group received 15 Dutch guilders (about $ 8.30) for their participation as a small incentive to encourage them 
to volunteer. The Leeuwarden subjects participating in the experiment were excused from an otherwise compulsory 
field trip. As all subjects were distributed randomly over the experimental conditions, it is assumed that the data of 
all three groups can be pooled without substantial loss of reliability. 

Sessions 

The experiment was administered in two sessions. In the first session, which took about two hours, the 
subjects worked with the experimental program and complete the post test. The second session took place between 
two and three weeks after Session 1, and is used to complete the retention test. This session takes about 30 
minutes. The second Velp group had to complete the retention test at home during the Christmas holiday, when the 
school was closed. They sent the test forms back by prepaid mail. For logistic reasons, the Leeuwarden group also 
sent the test forms back by mail. 

The first session took place in a classroom that was furnished for the occasion. Four sets of equipment 
were installed on tables along two walls. The minimum distance between the tables was two meters (about six feet) 
to ensure that each subject could work individually without being distracted by adjacent participants. Each set of 
equipment consisted of a computer with keyboard and mouse, a monitor, a video disk player, and an audio amplifier 
with headphones. The work of the subjects was monitored by one of the experimenters who had a table near the 
front of the classroom. 

The second session took place in the same room where Session 1 took place (first group in Velp), at home 
(second group in Velp), or in a different classroom (group in Leeuwarden). While the capacity of Session 1 was four 
subjects at a time, the retention tests were administered in larger groups. The delay between post test and retention 
test could therefore differ from two to three weeks. The random assignment of subjects to conditions was, however, 
such that no undesired influence on the data analyses is expected. On the average, the period between Session 1 and 
Session 2 was two-and-a-half weeks. 

The experimental video disk program 

The experimental program that was used is a computer-controlled interactive video disk about the industrial 
process of cheese making. The video disk program is divided in chapters. There is a main program of seven chapters 
that is used for data collection (total running time: 31.5 minutes), preceded by an introductory chapter with a length 
of 4.30 minutes 1 . The introductory chapter gives an overview of the cheese making process, which is used to 
familiarise the subjects with their experimental task and to orient them on the content of the main program. 

Conditions 

The program was prepared to serve two conditions: FIXED and FREE. In the FIXED condition, the 
subjects worked through all seven chapters in a linear order. The program presents the first video chapter without 
interruption, followed by questions about this chapter. All questions are open questions that require a brief sentence, 
a word or a number as an answer. After answering all questions about the chapter, the answers to the questions are 



*) European interactive video disks use the PAL video format which has a frame rate of 25 fps. One side of a video 
disk in CAV mode counts 54,000 frames, thus yielding 36 minutes of running video. 
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reviewed one by one. Correct answers are reinforced by a feedback statement that also contains the full text of the 
answer. For missed questions a small piece of video is repeated and the question is posed for a second time. This 
time feedback is provided that tells what the right answer is irrespective of right or wrong. When all questions have 
been reviewed, the subject had no other choice than continuing with the next chapter. In this way the subject worked 
through all chapters. The subject only controlled the pace of the session by taking more or less time for answering 
questions or for taking a pause between chapters. In the FREE condition, the subject was free to select any order of 
the chapters and could also decide to answer questions immediately or later, or even before watching the related video 
material. When watching video, the subject in the FREE condition could interrupt the chapters as often as he or she 
wants. In that way, video segments were defined that are (much) shorter than the complete chapters. After each 
interruption, the program offered the option to answer questions about the segment just watched; or about the chapter 
as a whole. Whatever the subject decided, the review system after answering questions was always the same as 
described in the FIXED condition. In both conditions, a subject could only leave the program if all questions from 
the program had been answered. This means that all subjects answered all questions that are embedded in the 
program. 

The FREE as well as the FIXED condition have been used in two versions to be able to research the effects 
of pre-instruction, resulting in four conditions in total: FREE with pre-instruction (FREE/P), FREE with no pre- 
instruction (FREE/NP), FIXED with pre-instruction (FIXED /P), FREE with no pre-instruction (FIXED /NP). 

Subjects who worked in the FREE/P and FIXED /P conditions received a manual with learning objectives. 
In this manual, the objectives were specified per chapter of the video disk program. The subjects were allowed to 
study the manual before they started working with the program. They could consult the manual during the 
experiment as often as they want. 

Tests, questionnaire and other material 

The post tests and retention tests that were used in all conditions contained 34 multiple choice questions, 
each with four alternatives. For every question in the post test there was a parallel question in the retention test. 
This was done to avoid plain recall of answers between post test and retention test. The data that were collected by 
these tests, are used to test all hypotheses except hypothesis H6. 

The tests were constructed using the questions that were used in the study by Breman (1995). For each test 
(the post test and retention test) 24 questions were taken almost verbatim from Breman, and 10 more questions were 
constructed in a similar style. The post test and retention test were nominally parallel. This means that empirical 
evidence for psychometric parallelism was lacking. The tests were, however, still completely similar. 

The post test and retention test each consisted of two parts. One part (24 questions) was directly related to 
the learning objectives from the pre-instruction conditions. The answers to these questions are used to test 
hypotheses H 1 , H3, H4, and H7 (related to the post test) and H2, H3, H5, and H7 (related to the retention test). The 
other part (10 questions) was not related to these objectives. The objectives- related part was meant to measure 
intentional learning, the other part to measure incidental learning. The difference is used to test hypotheses H8 and 
H9. Table 1 helps to explain how intentional and incidental learning is operationalised in this study. 



Table 1: Definition of relative scores YI to Y4 to test hypotheses H8 and H9 





relative score on 


Pre-instruction by means of 


24 learning- objective related 


10 non-leaming-objective related 


learning objectives: 


items 


items 


Yes 


YI (=abs.score/24) 


Y2 (=abs. score/ 10) 


None 


Y3 (=abs. score/24) 


Y4 (=abs .score/ 10) 



To be able to compare the test scores for the 24 learning-objective related items with the test scores for the 10 non- 
learning-objective related items, the absolute scores are normalised by dividing the scores by the number of 
questions: 24 (yielding the factors YI, Y3), respectively 10 (yielding the factors Y2, Y4) (Table 1). If pre- 
instruction has the predicted effect, YI should be larger than Y3 (Y 1>Y3): subjects who received pre-instruction have 
higher scores for intentional learning than subjects who did not receive pre-instruction. The effect of pre-instruction 
is also expected to cause Y2 to be smaller than Y4 (Y2<Y4): subjects who received pre-instruction directed their 
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attention primarily to learning-objective related content and paid less attention to other parts of the subject matter. 
Subjects who did not receive pre-instruction are expected to pay equal attention to all subject matter. This equally 
distributed attention should cause Y3 to be equal to Y4 (Y3=Y4). Lastly, Y1 is expected to be larger than Y2 
(Y1>Y2): directing attention to the learning-objective related content will yield a relatively better score on the 24 
learning-objective related items in the tests compared to the 10 non-leaming-objective related items. It is obvious 
that this reasoning only holds if the mean level of difficulty of the 24 learning-objective related items is equal to that 
of the 10 non-leaming-objective related items. The experimental design attempts to account for this by randomly 
selecting the 10 items for which no learning objectives are presented in the pre-instruction from all 34 items in the 
test. 

Directly after working with the program and before administering the post test, the subjects were asked to 
complete a questionnaire about the audio-visual qualities of the program, and their opinions about the program in 
general. This questionnaire was used to check whether the subjects were at ease when working with the program 
and did not encounter problems when using it. 

Each subject receives four blank sheets of paper (A4 format) and a pen for note taking during the 
experiment. Further, a sheet with instructions for the subject is on each table in a version that fits the experimental 
condition of that subject. 

Procedure 

For this experiment a 2x2 factorial design was used with the factors pre-instruction and locus of control. 
The subjects were randomly assigned to the four conditions. 

The answers of the subjects to the questions in the program were registered in a log file. In the FREE 
condition, the chosen segment lengths were also registered. The segment-length data are used to test H6. 

Data collection took place according to the following procedure: 

1 

(Only in conditions FREE/P and FIXED/P:) 

Reading the manual with the instructional objectives with the opportunity to ask the experimenter questions 
about the objectives. 

Practising with the instructional system while working through the introductory video chapter. 

Carrying out the experimental task. 

Completing the questionnaire about the audio-visual qualities of the program, and about working with the 
program in general. 

Completing the post test. (End of Session 1.) 

Session 2 

Step 6 After about two-and-a-half weeks: completion of the retention test. 



Results 

The results of the experiments are as follows. Table 2 shows the mean scores on the total post test of 34 
items, the standard deviations and the number of observations per condition. Table 3 shows the same data for the 
retention test. In both tables, the numbers between brackets show the data for the subset of 24 learning-objective 
related items. 



Session 
Step 1 



Step 2 
Step 3 
Step 4 

Step 5 





Table 2: Post test results for mean (M), standard deviation (SD) and number of 

observations (n) 





Condition 






FIXED 

(computer control) 


FREE 

(learner control) 


Total 


Pre-instruction 


M=28.58 (19.83) 
SD= 1.88 (1.85) 
n=12 


M=26,38 (18.69) 
SD=3.28 (2.81) 
n=13 


M=27 .44 (19.24) 
SD=2.87 (2.42) 
n=25 


No pre-instruction 


M=27.53 (18.47) 
SD=2.20 (1.30) 
n=15 


M=26.36 (18.27) 
SD=2.29 (1.85) 
n=l 1 


M=27 .04 (18.38) 
SD=2.27 (1 .52) 
n=26 


Total 


M=28.00 (19.07). 
SD=2.09 (1.68) 
n=27 


M=26,38 (18.50) 
SD=2.81 (2.38) 
n=24 


M=27.24( 18.80) 
SD=2.57 (2.04) 
n=51 


Between brackets: Data for the subset of 24 learning-objective related items. 




Table 3: Retention test results for mean (M), standard deviation ( SD) and number of observations ( n) 




Condition 






FIXED 

(computer control) 


FREE 

(learner control) 


Total 


Pre-instruction 


M=20.70 (14.10) 
SD=4.71 (3.87) 
n=10 


M=19.27 (13.27) 
SD=3.58 (2.80) 

n=l 1 


M=19.95 (13.67) 
SD=4.12 (3.29) 
n=21 


No pre-instruction 


M=21.33 (14.58) 
SD=3.45 (2.15) 
n=12 


M=20.56 (13.89) 
SD=5.03 (4.11) 
n=9 


M=2 1.00 (14.29) 
SD=4.1 0(3.07) 
n=21 


Total 


M=2 1.05 (14.36) 
SD=3.98 (2.98) 
n=22 


M=19.5 (13.55) 
SD=4.22 (3.36) 
n=20 


M=20.48 (13.98) 
SD=4.09 (3.16) 
n=42 



Between brackets: Data for the subset of 24 learning-objective related items. 



Table 4 shows the results of a two-way analysis of variance of the data. The results show a significant main 
effect for locus of control for the complete post test with a level of significance of .020: Subjects in the FIXED 
condition performed better than in the FREE condition. The related effect size is 1 .62 points on a scale of 34 points 
or 4.8 percent. These results confirm H4. No further significant effects appeared to occur, which means that there is 
no sufficient evidence to support HI, H2, H3,and H5. 
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Table 4: ANOVA results for the post test and the retention test 

MS F 



Significance of F 



Post test: 



Main effect for locus of 
instructional control 



35.518 (5.541) 1 5.791 (1.368) 



0.020 (0.248) 



Main effect for pre- 
instruction 



4.021(10.679) 1 0.656(2.637) 



0.422 (0.111) 



Interaction effect 



3.332 (2.822) 1 0.543 (0.697) 



0.465 (0.408) 



Retention test: 

Main effect for locus of 12.687 (6.019) 1 0.729 (0.573) 

instructional control 

Main effect for pre- 9.239(3.108) 1 0.531 (0.296) 

instruction 



Interaction effect 



1 .095 (0.046) 



1 



0.063 (0.004) 



Between brackets: Data for the subset of 24 learning-objective related items. 



0.399 (0.454) 
0.471 (0.590) 
0.803 (0.948) 



To test whether subjects in the FREE/P condition choose shorter video segments than in the FREE/NP 
condition, a t-test for independent samples was used. Table 5 shows that there are no significant differences for self- 
chosen segment length, which yields no support for H6. Given the fact that the mean segment length in the FIXED 
condition is 30.86 information elements, Table 5 also shows that the subjects decided, on the average, for segments 
that are about 80 percent of the length of the pre-defined chapters. A closer look at the data reveals that only 3 
subjects divided the segments into very small portions (with an average length of about 10 information elements), 
while the mean segment length of the remaining group is 26.65 information elements or 86% of the maximum 
possible mean segment length (Figure 1). 



$ 

Table 5: T-test for differences in mean self -chosen segment length 





n 


M 


SD 


df 


t-value 


p- value 


Pre-instruction 


13 


24.58 


5.74 


14.79 


.16 


.875 


No pre-instruction 


10 


24.07 


8.71 









4 * 

: Segment length was measured as a number information elements, see text next to the list of 
hypotheses. 



The questionnaire that was administered just after working with the program and before the post test yielded, in 
general, positive results about the appreciation of the audio-visual and interactive qualities of the program. 

Table 6 and Table 7 show whether differential results occurred with respect to intentional and incidental learning, 
both for the post test and the retention test. Table 6 compares differences in intentional and incidental learning 
between the pre-instruction and no-pre-instruction conditions. Table 7 compares intentional and incidental learning 
within the pre-instruction and no-pre-instruction conditions. Only Table 7 shows a significant result: Subjects in 
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the no-pre-instruction condition performed better (pc.001) on the 10 test items that relate to incidental learning for 
the pre-instruction group (with a mean score of 86 percent, see Table 6) than on the 24 items that relate to 
intentional learning of the pre-instruction group (with a mean score of 77 percent). This means that neither H7 or 
H8 are supported, while the reported significant effect was not expected. 
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Figure 1: Distribution of self-chosen segment length in the FREE condition. 



Table 6: T-tests for differences between Y1 and Y3 and for Y2 and Y4 for post test and retention test 







Variable 


n 


M 


SD 


df 


t-value 


p- value 


Difference 


Y1/Y3, 


Y1 


25 


.802 


.101 


40.20 


1.50 


.141 


post test 




Y3 


26 


.766 


.064 








Difference 


Y2/Y4, 


Y2 


25 


.820 


.126 


48.99 


-1.27 


.210 


post test 




Y4 


26 


.865 


.129 








Difference 


Y1/Y3, 


Y1 


21 


.569 


.137 


39.80 


-.63 


.532 


retention test 




Y3 


21 


.595 


.128 








Difference 


Y2/Y4, 


Y2 


21 


.614 


.139 


36.87 


-1.12 


.269 


retention test 




Y4 


21 


.671 


.187 
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Table 7: T-testfor differences between paired samples Y1 and Y2 and Y3 and Y4 for post test and retention 

test 





n 


M 


SD 


df 


t-value 


p-value 


Y2-Y1; post test 


25 


.0183 


.150 


24 


.61 


.548 


Y4-Y3; post test 


26 


.0994 


.126 


25 


4.01 


.000 


Y2-Y1; retention test 


21 


.0448 


.113 


20 


1.83 


.083 


Y4-Y3; retention test 


21 


.0762 


.188 


20 


1.86 


.078 
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Discussion 

The results of the study show a significant result for locus of control. Subjects in the FIXED condition, in 
which the order and length of video segments was controlled by the computer, performed better on the post test than 
in the FREE condition, in which they were allowed to interrupt video segments as often as they wanted and in which 
they were free to go through the material in any order (Table 4). The effect size of 1 .62 points (Table 2) on a scale 
of 34 points or 4.8 percent is, however, not impressive. Still, this result is in line with findings by Breman (1995), 
who also obtained a small but significant difference in test performance between a free and a fixed condition in favour 
for the fixed condition. 

Breman used the same video disk and the same computer program to control it in a pre-test post-test design. 
In Breman's case, the mean of mean self-chosen segment length in his free condition was 21.55 Information 
Elements (in time: about 3 minutes). This led to the reasoning that the longer segments of the fixed condition 
yielded better results because longer watching is a more difficult task, and that the subjects were sufficiently aware of 
this to decide to invest a greater amount of mental effort (AIME: Salomon, 1984) than in the case of the free 
condition where the subjects could interrupt whenever they felt like it. These results were similar to the results of 
Verhagen (1992b). Also Verhagen found that forced exposure to of relatively long segments supports the learning of 
factual information. 

In the present study, however, the differences between mean self-chosen segment length and the mean 
segment length in the FIXED condition are rather small (Figure 1). An alternative explanation should be considered. 
It could be that the freedom of selecting the order and length of video segments in the FREE condition formed a 
burden for the subjects, who were aware of the need to make decisions by themselves all the time, while the subjects 
in the FIXED condition could concentrate on the content of the programme, guided by the computer. Further 
research around the AIME concept is recommended, to try to find an answer to this problem. 

Table 7 shows the unexpected effect that subjects in the no-pre-instruction condition performed better on test 
items that relate to incidental learning in the pre- instruction condition than on test items that relate to intentional 
learning in the pre-instruction condition. Without the orienting objectives from the pre-instruction, no differences in 
performance should be expected. It is a fact that the 10 questions for incidental learning were constructed for this 
experiment, while the other 24 questions were already there from the experiments by Breman. Although no cues in 
this respect were initially there, it still could be the case that the difficulty level of the 10 questions is lower than the 
difficulty level of the other questions. If that should be true, then the fact that in the pre- instruction condition no 
differences were found should be an indication in favour of H7. As things are now, no possibilities exist to arrive at 
a clear conclusion about the use of pre-instruction. 

The fact that pre- instruction makes no difference may, however, also be caused by the quality and structure 
of the audiovisual program. It is possible that the structure and the clear presentation of the audio-visual material 
and the questions of programme overruled the effects of pre-instruction (Mayer, 1979; Hannafin, Phillips, Rieber, 
Garhart, 1988). 

In all, only one main effect results: In this study, program-control yielded a better post test performance 
that learner control, although the effect size is rather small. Given similar results by Breman (1995), Verhagen and 
Breman (1995), Verhagen (1992), and Zijderveld (1991), it can be recommended that for learning factual information 
it seems to be a matter of taste and contextual conditions whether program-controlled or learner controlled interactive 
programs should be used. And if all other things were equal, the easier (and thus cheaper) to be produced program- 
controlled programs could be preferred. 

As an explanation, it could be argued that if the audio-visual presentation is a clear story, maybe the learner 
control options are just a burden to the learners. Clarity may also cause the limited need for selecting short 
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segments, as appeared from the data. (Figure 1). 

It should be noted that in this research and in other research mentionend, the subject matter is about factual 
information only. Still, it seems warranted to recommend the following for future reseach: 

May be it is time to revive the attention for story-based media, to balance the overwhelming 
attention currently given to interactive media. 
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Abstract. This study involved student teachers in the two public universities in the United 
States and was designed to investigate the use of computers by student teachers in their 
practicums: the manner and frequency of computer use, student teachers' perception of their 
training, their attitudes toward the role of the computer in teaching, and factors that might be 
associated with student teachers' use of computers. The study yielded both postive and 
negative results and pointed to the need of restructuring IT programs in the two univeristies. 

Literature Review 

Research indicates that school teachers make little use of computers (Sheingold & Hadley, 
1990). Teachers often cite lack of computer skills as one of the major reasons for not using 
computers. To ensure that new generation of teachers will not simply add to the pool of 
teachers who need computer training once they are on job, teacher education programs in 
universities and colleges have started to offer educational computing courses in the hope that 
their newly-graduates will successfully implement the computer technology in teaching 
practice. 

One of the goals of educational computing courses is to provide computer skills to preservice 
teachers. This approach makes sense considering the fact that often students enter teacher 
education programs with little or no computer skills. In Summers' study (1988), 74% of the first 
year students of elementary education majors had little or no previous computer experience. 
Reed, Ervin, and Oughton (1995) conducted a longitual study evaluating the computer 
experience of new entrants to a teacher education program. Their study showed that during a 
ten-year period from 1985 to 1994, there was an increase in computer experience from students 
who entered the program early and students who entered the program more recently. However, 
compared with English majors and mathematics education majors, elementary education majors 
had less computer experience and higher computer anxiety. 

Another goal of educational computing courses is to foster positive attitudes among preservice 
teachers toward computers and develop their awareness of the value of the computer's role in 
education. One's attitudes toward computers play a significant role in the successful 
implementation of computers in educational settings (Koohang, 1987). Computer attitudes often 
refer to one's gut feelings toward the computer such as computer anxiety, one's opinion about the 
computer’s role in teaching and learning, and one’s perception of the adequacy of his/her 
training as a confident and competent computer user. Summers' study (1988) showed that 40% of 
students started college with negative feelings toward computers. Koohang’s study (1987) 
demonstrated a positive relationship between one's computer attitudes and one's computer 
experience. His study (1987) also showed that students who had more computer experience had 
more positive attitudes toward the computer. The more computer experience a student had, the 
lower became his/her computer anxiety. 

If one’s computer attitudes can be influenced by one's computer experience, a computer training 
course can be used as an intervention in improving students' attitudes. Savenye and Orr’s study 
(1992) described the effect of a computer training course on students' attitudes. The subjects in 
the study participated in a semester-long computer literacy course that significantly improved 
their attitudes. 

However, Boone and Gabel's study (1994) issued a caution that computer training courses do not 
necessarily lead to positive attitudes toward computers. The subjects in their study (1994) were 
science education majors. The students' attitudes toward computers were monitored across a 
two-year period as they progressed through the teaching training program. During this period, 
the students had opportunities to take a three-credit-hour introductory computer course as a 
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part of degree requirement as well as other optional computer courses. The findings of the study 
indicate that the students’ attitudes became less positive toward computer use. The 
implication of this research is that in order to facilitate positive attitudes, it is insufficient to 
merely offer students computer courses. What matters is the content of the course and how the 
course is taught. As Koohang found in his study (1987), the students’ attitudes also relate to the 
nature of their computer experience. 

When the students completed their computer training courses, what beliefs and values did they 
formulate regarding the computer's role in education? What were the students' perceptions of 
their preparation of computer use? Byrum and Cashman's study (1993) endeavored to answer 
these questions. Their study found that 83% of the students at six Midwest universities felt 
they were prepared to integrate computers into the curriculum. However, a detailed 
breakdown revealed that the majority of the students preferred using computers as a 
supplement to their teaching with drill and practice as their first choice. This indicates that 
these students did not break through the traditional view of the computer's role in education. 
The result was not a surprise since only 24% of the students had had a chance of developing 
lesson plans integrating computers outside computer training courses. The researcher 
emphasized the need of modeling the uses of computers for teaching and learning by faculty 
who teach education courses. 

Even when students feel they are prepared for the computer use, will they use the computer once 
they are placed in classrooms? There is a sizable gap between the expected level of use by 
preservice teachers and the level of computer use by practicing teachers (Marcinkiewicz, 1994- 
1995). Practicing teachers had a much higher incidence of non computer use, compared to their 
preservice counterparts. The researcher attributed this to the fact that preservice teachers 
received computer training in their teaching training program. On the other hand, it might 
indicate that "future teachers' expectation of computer use will become lowered by external 
circumstances or those over which they have no control unless they are extraordinarily 
motivated" (p. 194). Both groups rated low in integrating the computer into the curriculum. 

Marcinkiewicz's speculation (1994-1995) was confirmed by the experience of newly-graduates. 
Novak & Knowles (1991) and Oliver (1994) focused on beginning teachers' computer use and 
found that even when these teachers received training in computer use while they were in the 
program, their computer use rated low. These beginning teachers "were only able to utilize the 
computer in somewhat limited way" (Novak, 1991; p. 50). They viewed the computer as an 
extra activity rather then an integral part of the curriculum. "There is little evidence, 
however, that the teachers selected programs which provided relevant practice for the 
students. Many of the programs they used demonstrated weak instructional design and were 
only remotely related to the existing curriculum" (Novak, 1991; p. 48). 

In Oliver's study (1994), a considerable number of beginning teachers were found not using 
computers even though in most schools, hardware and software access was not a problem. Those 
who had completed educational computing courses were no more likely to use the computer than 
those who had not. The researcher also found that the computer use patterns reflected the 
nature of the instructional programs. The primary beginning teachers made significantly more 
use of computers as an instructional aid due to their exposure to a curriculum that emphasized 
classroom implementation of the computer, whereas the secondary beginning teachers made 
more use of computers as a personal and management tool, which reflected their curriculum 
emphasis. 



In summary, the review of the literature pointed to the fact that the nature, not the amount, of 
educational computing courses is the key factor determining the effectiveness of computer 
training of preservice teachers and influencing preservice teachers' future implementation of 
computers in educational settings. 

In order to offer educational computing courses that cater to students’ need, it is of vital 
importance to assess the effectiveness of the existing computer training courses so that these 
courses can be restructured to "embrace curriculum applications, strategies, and issues" (Oliver, 
1994; p. 87). 

The present study focused on student teachers' use of computers as an assessment of the 
effectiveness of the computer training in teacher education at two universities. The study 
focused on student teachers rather than on beginning teachers’ use of computer because beginning 
teachers are under a tremendous amount of pressure trying to cope with their early days in the 
classroom. "Their computer use was notably impacted by their first-year status" (Novak & 
Knowles, 1991; p. 49). This pressure is lessened with student teachers for they can count on 
guidance and support from university supervisors and classroom cooperating teachers. 

Therefore, the data generated from investigation of their computer use during practicum reflect 
a truer picture of their computer training in teacher education. 

The student teaching is a crucial period for student teachers. "It is in these experiences that 
education majors become acquainted with the realities of life in elementary and secondary 
classrooms, look for real-world connections to content presented in their university foundations 
and teaching methods classes, and develop their instructional and managerial skills" (Hunt, 
1995; p. 37). Their experiences in this period help to shape their future teaching style. 
Consequently, it is important to investigate teaching practicums so that appropriate strategies 
can be developed to aid student teachers in integrating computers into their teaching. 

However, the teaching practicum is an overlooked area in educational computing research. 

Few studies examine this crucial period in terms of student teachers’ use of computers. Student 
teachers have been the focus of two recent studies. One was conducted by Dunn & Ridgway 
(1991a, 1991b) in the United Kingdom and one was conducted by Downes (1993) in Australia. 
Dunn & Ridgways’ research (1991a, 1991b) focused on the pattern of the computer use among the 
practicum students. He conducted his research when the students completed their first teaching 
and final teaching respectively. Downes' research (1993) focused on student teachers’ computer 
use with children and related factors affecting use. Up to date, no research has been located in 
the United States that focuses on student teachers’ use of computers during their practicum. 

This study involved student teachers in the United States and was designed to investigate the 
use of computers by student teachers in their practicums: the manner and frequency of computer 
use, student teachers' perception of their training, their attitudes toward the role of the 
computer in teaching, and factors that might be associated with student teachers' use of 
computers. 

Background 

Teacher education programs in the two public universities in this study shared similar 
characteristics. In both universities, elementary education majors are required to take a three- 
credit computers in education course. The course teaches computer literacy as well as classroom 
applications of educational software and tool software. The course is an option for secondary 
education majors. Students complete all the course work before they start a semester-long 
teaching practicum. 
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Methodology 

The sample population consisted of all the practicum student teachers in two universities 
during the fall semester, 1995. The total sample population for this study was 120. A 
questionnaire was developed containing items related to each of the research questions. The 
questionnaire contained 23 questions including yes/no, multiple choice and Likert-type 
questions. The researchers distributed the questionnaires among supervising teachers and they, 
in turn, gave the questionnaires to the students they were supervising during the final week of 
the semester. One hundred and ten questionnaires were collected with a response rate of 92%. 

Results 

Background Information 

Background information on the subjects included gender, age, prior computer experience, home 
computer ownership, and types of school where they did their student teaching. Background 
information is presented in table 1. As always the case among education majors, a great number 
of the subjects were females (80%). The majority of the students (69%) were ages 20-25. 

Students’ computer experiences were varied. Surprisingly, about one fifths of the students 
(21%) had never taken any computer courses at the university level. Sixty-four percent of the 
students reported that they learned the computer in high school. Less than half of the students 
(45%) owned a home computer. Elementary schools were where most student teachers did their 
practicum (66%). Other types of schools (8%) referred to preschools, junior high or K-12 
schools. 





Table 1. Background Information 


What is your gender? 






Male 


n=22 


20% 


Female 


n=88 


80% 


What is your age in years? 






20-25 


n=76 


69% 


26-30 


n=17 


16% 


30-35 


n=5 


5% 


over 35 


n=12 


11/7/ 

Al/O 


Did you learn to use the computer in high school? 




Yes 


n=68 


64% 


No 


n=38 


36% 


How many computer courses have you had in the university? 




None 


n=23 


21% 


One 


n=5l 


46% 


Two 


n=18 


16% 


Over Three 


n=18 


16% 


Do you own a personal computer? 




Yes 


n=49 


45% 


No 


n=59 


55% 


In what type of school did you student teach? 




Elementary 


n=72 


66% 


Middle 


n=5 


5% 


High 


n=24 


22% 


Other 


n=9 


8% 
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School Computing Environment 

The computing enviroiiment in practicum schools is shown in table 2. In most schools, computers 
were located in the computer lab (48%). Over half of the student teachers (56%) reported 
seeing their supervising teachers using the computer in the classroom. An overwhelming number 
of the students (83%) saw other teachers using computers in their practicum schools, an 
indication that computers are a common scene in schools. 



Table 2. Computing Environment in Practicum Schools 
How were computers placed in the school? 



one computer shared between 
classroom 


n=5 


5% 


one computer in each classroom 


n=14 


13% 


computers in a lab 


n=53 


48% 


combination of above 


n=33 


30% 


do not know 


n=3 


3% 



Did you see your supervising teachers use the computer in the classroom? 

Yes n=61 56% 

No n=49 44% 

Did your supervising teacher talk to you about using the computers in the classroom? 
Yes n=64 60% 

No n=42 40% 

Did you see other teachers use computers? 

Yes n=91 83% 

No n=19 17% 

Did other teachers talk about using computers in their classroom? 

Yes n=59 54% 

No n=36 46% 



Computer Use during the Teaching Practicum 

A high percentage of the students ( 81%) used computers during the practicum with 64% 
reporting their use was over 10 times (see table 3). Fifty-one percent of the students used the 



Table 3. Com outer Use d urine the Teachine Practicum 


Did you use computers while student teaching? 




Yes 


n=94 


85% 


No 


n=16 


15% 


How often did you use computers (times)? 




Only once 


n=l 


1% 


2-5 times 


n=16 


17% 


6-10 times 


n=17 


19% 


More than 10 times 


n=60 


64% 


How did you use computers? 






personal use 


n=29 


31% 


used with children 


n=17 


18% 


used both personal 


n=47 


51% 


and with children 
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A detailed breakdown revealed that drill and practice was the most commonly used type of 
educational software (50%) with multimedia rating the lowest (13%). Word processing was 
the tool software used by most of the student teachers (67%) with database rating the lowest 
(11%). Only a small number of the student teachers used computers for telecommunication 
(10%). 



TableA. Types of Software Used during the Practicum 


What types of educational software did you use 


while student teaching? 


drill and practice 


n=55 


50% 


game 


n=36 


33% 


problem solving 


n=35 


32% 


tutorial 


n=30 


27% 


simulation 


n=17 


16% 


multimedia 


n=14 


13% 


authoring 






What types of tool software did you use while student teaching? 


word processing 


n=74 


67% 


graphics 


n=40 


36% 


spreadsheet 


n=15 


14% 


database 


n=12 


11% 


telecommunication 


n=ll 


10% 



For those who did not use computers during the practicum, 9 students cited the major reason as a 
lack of access to hardware (45%), 7 students cited lack of access to software (35%) and 2 students 
mentioned that they lacked confidence or computer skills. The rest of the students did not 
respond to the question. 



Table 5. Reasons for not Using Computers 



If you did not use computers, why? 
lack of confidence 
lack of computer skills 
lack of access to hardware 
lack of access to software 



n=2 


10% 


n=2 


10% 


n=9 


45% 


n=7 


35% 



Student Teachers' Attitudes and Perception 

The student teachers in this study had positive attitudes toward the role of the computer in 
education with 43% of the students considering it very important, 40% important and 17% 
somewhat important (see table 6). None of the students considered computers unimportant. 
Ninety-five percent of the students agreed or strongly agreed that computers improved their 
teaching effectiveness. Only 6% of the student teachers felt they were not prepared for 
computer use in future teaching. All of the students indicated that there was a chance that 
they were going to use computers in teaching if access to hardware and software were not a 
problem. 
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Table 6. 


Student Teachers' Attitudes and Perceptions 


How important do you feel computers are in teaching? 




not important 


n=0 


0% 


somewhat important 


n=18 


17% 


important 


n=44 


40% 


' very important 


n=47 


43% 


Do you agree that your teaching effectiveness is unproved through the use of the computer? 


Strongly disagree 


n=2 


2% 


Disagree 


n=3 


3% 


Agree 


n=62 


62% 


Strongly agree 


n=36 


33% 


How well do you feel you are prepared to use computers? 




not prepared 


n=6 


6% 


somewhat prepared 


n=35 


32% 


prepared 


n=39 


36% 


well prepared 


n=29 


27% 


If hardware and software are 


available, how likely would you be willing to use the computer 


in your future teaching? 






not likely 


n=0 


0% 


somewhat likely 


n=16 


15% 


likely 


n=31 


28% 


most likely 


n=62 


57% 



Factors Associated with Computer Use 

The present study found no gender difference in computer use. Male and female student teachers 
alike tended to use computers while student teaching (82% versus 82%) 

(see table 7). However, due to the imbalance of numbers between male (20%) and female (80%), 
a conclusion cannot be made. The student teachers who were in the 20-25 age group were most 
likely to use computers during the practicum (88%). Again, due to the imbalance of numbers in 
age groups, the relationship between age and computer use can not be determined. There was not 
much difference in computer use between those who learned computers in high school and those 
who did not. The percentage was close (82% versus 79%) in terms of computer use for both 
groups. The relationship between computer use and computer courses taken at university level is 
puzzling. The number of student teachers who used computers without taking any computer 
courses at university level was close to those who took three computer courses at university 
level (77% versus 79%). However, there was a high percentage of computer use among the 
student teachers who took two computer courses (89%) and those who took one computer course 
(82%). Ninety-two percent of student teachers who owned computers at home used them in the 
practicum compared to 73% who did not own a computer. Elementary school student teachers 
were most likely to use computers (84%). However, there were only 5 student teachers at 
middle school level. This fact needs to be considered in reaching a conclusion. 



IflbJe.Zi Effect of Gender Age, Co m p ute Lgxpe rien ce, H o m e C o m pu tin g a n d Ty p e s of Practic um 

Schools on the Use of Computers 

Use Computers during the 
practicum 



Gender 


n 


percent 


n 


percent 


Male 


22 


20% 


18 


82% 


Female 


88 


80% 


70 


82% 


Age group 


20-25 


76 


69% 


66 


88% 


26-30 


17 


16% 


9 


56% 


30-35 


5 


5% 


4 


80% 


Over 35 


12 


11% 


9 


75% 


Learned computers in high 
school 


Yes 


68 


64% 


54 


82% 


No 


38 


36% 


30 


79% 


Computer courses taken at 
university level 


None 


23 


21% 


17 


77% 


One 


51 


47% 


41 


82% 


Two 


18 


16% 


16 


89% 


Over Three 


18 


16% 


14 


78% 


Personal computer ownership 


Yes 


49 


45% 


45 


92% 


No 


59 


55% 


43 


73% 


Types of schools while student 
teaching 


Elementary 


72 


66% 


63 


84% 


Middle 


5 


5% 


4 


80% 


High School 


24 


22% 


18 


75% 


Others 


9 


8% 


7 


78% 



When supervising teachers used computers or talked about computer use, approximately 95% of 
the student teachers used computers in the classroom (see table 8). Whereas, only 64% of them 
did so when the supervising teachers did not use or discuss computer use. When other teachers 
in the practicum school used or talked about computers, approximately 85% of the student 
teachers used computers and only 68% of them used computers when the other teachers did not 
use or discuss computer use. For those who thought computers were very important or important 
in teaching, approximately 87% used computers in their practicum, whereas only 61% did so 
when they thought computers were somewhat important (see table 9). Among the student 
teachers who strongly agreed that their teaching effectiveness was improved by the use of 
computers, 94% of them used computers, compared with 67% of computer use among those who 
disagreed. Two students strongly disagreed that computer use improved their teaching 
effectiveness. Both of them used computers while student teaching. However, both students 
considered computers important in teaching. Ninety-one percent of the student teachers 



reported computer use in schools where computers were placed both in the lab and in the 
classroom, while 78% of the student teachers reported using computers in schools where 
computers were placed only in the lab. However, the placement of a computer in a classroom 
did not guarantee computer use. Four student teachers did not use computers even they reported 
they had a computer placed in their classroom. Five students reported that computers were 
shared between classrooms. All five students used computers during their practicum. 



Table 8. Effect of Computing Environment in Practicum Schools on Computer Use 










Use computers during the 












practicum 






n 


percent 


n 


percent 


Supervising teacher’s use of 


Yes 


61 


56% 


58 


95% 


computers 


No 


49 


44% 


30 


61% 


Supervising teacher talk 


Yes 


64 


60% 


60 


94% 


about the use of computers 


No 


42 


40% 


26 


62% 


Other teachers' use of 


Yes 


91 


83% 


76 


84% 


computers? 


No 


19 


17% 


12 


63% 


Other teachers' talk about 


Yes 


59 


54% 


51 


86% 


using computers 


No 


36 


46% 


24 


67% 


Placement of computers in 
practicum schools 




n 


percent 


n 


percent 


One computer between 
classrooms 




5 


5% 


5 


100% 


One computer in each 
classroom 




14 


13% 


10 


72% 


Computer lab 




53 


48% 


40 


78% 


Combination of the 
above 




33 


30% 


39 


91% 



Iflhte-iLJfJfect o f St uden t T eachers’ sJVttitudeJifldJEfircepliQn 



Use Computers during 
the practicum 





n 


percent 


n 


percent 


How important do you feel computers are 
in teaching? 


Very important 


7 


43% 


40 


87% 


Important 


44 


40% 


37 


86% 


Somewhat important 


18 


17% 


11 


61% 


Not important 


0 


0% 


0 


0% 


Was your teaching effectiveness improved 
by computer use ? 


Strongly disagree 


2 


0% 


2 


100% 


Disagree 


3 


15% 


2 


67% 


Agree 


62 


28% 


50 


76% 


Strongly agree 


36 


57% 


33 


94% 



Discussion 

This study yielded both positive and negative results. The student teachers in this study had a 
higher percentage of computer use than those in Dunn & Ridgway's study (1991b) and Downes’ 
study (1993). Eighty-five percent of the student teachers in this study reported using computers, 
compared with 71% of computer use by those in Dunn & Ridgways’ study. More student teachers 
used computers with children compared with those in Downes' study (1993). Fifty-one percent 
of the student teachers used computers with children and for personal purposes, whereas, only 
20% of the student teachers did so in Downes' study (1993). 

Student teachers held positive attitudes toward the role of computers in education. Teachers’ 
attitudes are of vital importance in the successful implementation of computers in educational 
settings (Koohang , 1987). Student teachers in this study acknowledged the value of computers 
in education and were willing to implement computers in their future teaching. 

However, the discovered pattern of computer use was far from satisfactory. Drill and practice 
comprised the highest usage rate in educational software. There was a low usage rate in game, 
problem solving, tutorial and simulation. Word processing had the highest usage rate of tool 
software. The usage rate dropped sharply for graphics, spreadsheet and database. While 
only 13% of the student teachers used computers for multimedia, just 10% of them used 
computers for telecommunication. This pattern of computer use suggested that these students 
were only being prepared for computer use in a limited way. Even though most of them 
considered themselves prepared, there was a discrepancy between their perception and reality. 

The findings of the study pointed to the need of restructuring the educational computing courses 
in teacher education at the two universities. At the present time, only one educational 
computing course is offered in teacher education at both universities. The course is a mixed 
approach of computer literacy and computer classroom applications. However, since students 
usually entered the course with little computer experience, they spent most of the class time 
learning how to operate the machine. Computer applications in teaching comprised only a 
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small segment of the course. The course might get students to the level of utilizing computers as 
a personal production tool, but not to the level of integrating computers into classroom teaching. 
We cannot assume that once students know how to use the computer, they will ultimately 
implement it in classroom teaching. Integrating computers into teaching is a complex process 
and students need to go through thorough training to develop necessary skills and strategies. 

Therefore, there is a need for a course focusing on strategies in applying computers to classroom 
teaching. However, the mere offering of the course will not achieve the goal. Research shows 
that the key factor affecting computer use is the nature of the course. Educational computing 
courses should cater to students’ needs and aim at preparing students for implementing 
computers in real classroom settings. 

Educational computing courses need to present students with examples and models of computer 
use in real classroom settings. "Our challenge is to provide a clear vision of how computers and 
technology can transform classroom instruction" (Wetzel, 1993; p. 335). Whenever possible, 
instructors should invite classroom teachers to demonstrate their experience of integrating 
computers into the curriculum. Not only should students learn how to teach by using the 
computer, they also need to experience learning the course content by using computers 
themselves as learners to feel the impact of the computer as a powerful learning tool. 

Whenever appropriate, instructors should present course materials by using computers 
themselves to model the computer's role as a teaching tool. "Teacher educators will do well to 
model the use of computers in instruction so as to provide realistic examples from which these 
future teachers can later build" (Novak & Knowles, 1991; p. 49). 

Computers will not exert a powerful influence if their use is confined to only educational 
computing courses. Computers need to be integrated across the curriculum in teacher education 
programs. Instead of utilizing computers only in educational computing courses, students should 
be both encouraged and required to use the computer outside the educational computing courses. 
This requires the cooperation and communication among faculty members in planning and 
offering courses. Faculty training workshops and seminars are essential in the program 
planning so that faculty can develop appropriate attitudes and necessary skills to accomplish 
the goal. 

In order to bridge the gap between the university classroom and the real world, teacher 
education programs should make efforts to seek connections with the real world. Field 
experience offers such a connection and should be an integrated component in educational 
computing courses. For example, opportunities should be created for students to visit school 
computer labs and observe computer usage in real classroom situation. 

The teaching practicum offers both intensive and extensive field experience. Since this period 
is crucial in shaping student teachers’ future teaching styles, technology should be made an 
integrated part of this experience. Dugdale (1994) reported a positive experience integrating 
computers into student teaching. The students were required to design classroom-based projects 
and implement their projects in the classroom setting while they were student teaching. This 
study shows that student teachers tend to role model their supervising teachers. It would 
promote student teachers' use of computers if they were assigned to supervising teachers who 
use computers in classroom teaching. Since other teachers might also have influence on student 
teachers, the entire school environment should also be taken into consideration for assigning 
practicum students in order to maximize computer use in instruction. 
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Teacher education programs should consider developing plans to train school teachers to help 
create and maintain an encouraging and supportive practicum environment to utilize computers 
in instruction. "Professional development is necessary for everyone in education to keep them up 
to date and refresh their approach (Davis, 1993; p. 239). Often, the cooperating classroom 
teachers lack appropriate attitudes or necessary computer skills to be supportive of student 
teachers' use of computers (Hunt, 1995). Davis (1993) quoted a promising model of providing in- 
service training to practicing teachers. The training was conducted at the school site and 
involved both practicing teachers and student teachers. Practicing teachers and students 
teachers took turns receiving training and then taking care of the class when the other one was 
receiving the training. Sometimes, the trainer was one of the practicing teachers or one of the 
student teachers. 

Home computer ownership is an important factor influencing the use of computers. This might 
be due to the fact the home ownership provides opportunities for the students to practice their 
computer skills. Some universities have made it an requirement that every student purchase a 
personal computer. It is speculated that such a requirement, in the long run, could help increase 
the computer literacy of students and promote the use of computers in teaching and learning. 

Conclusion 

This study focused on student teachers at two universities in the United States and provided an 
insight into the student teachers' use of computers during the practicum and factors associated 
with the use. The findings of the study highlighted the need to restructure the educational 
computing courses in the teacher education programs at these two universities. 

"An essential part of integrating technology into the educational curriculum is the training of 
persons who will ultimately use the technology" (Byrum & Cashmen, 1993; p259-260). Teacher 
education programs need to consider infusing technology into student teaching, the crucial 
period in students’ professional development, so that students will be better prepared to 
implement computers in real classroom settings. 
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Research has often found little or no significant differences in 
over - all performance for math students using graphing calculators 
compared to traditional (non-calculator) classes (Scott, 1994; Tolias, 
1993) . However, when performance is divided into procedural and 
conceptual levels, significant differences show up at the conceptual 
level (Tolias, 1993) . What factors relate to mathematical performance 
with graphing calculators at the conceptual cognitive level? Factors 
strongly suggested for consideration are gender, spatial 
visualization, mathematical confidence, basic mathematical ability, 
and classroom graphing calculator utilization. 

Background 

Two levels of mathematical understanding are defined by 
researchers in the field of mathematical learning. Hiebert and 
LeFevre (1986) divide mathematical knowledge into procedural and 
conceptual. Procedural knowledge is "familiarity with the symbol 
representation system and rules, algorithms, and procedures" (pg. 9), 
while conceptual knowledge is "a connected web of knowledge, a network 
in which the linking relationships are prominent as the discrete 
pieces of information" (pg. 3-4). For Hiebert and LeFevre, 
instruction should foster conceptual knowledge construction. Richard 
Skemp ' s (1987) cognitive learning theory delineates two types of 
understanding — instrumental and relational. For mathematics, 
instrumental understanding is the application of a rule or procedure, 
while relational understanding involves relating a task to an 
appropriate schema (knowing both how and why in problem solving) . For 
Skemp, relational understanding and conceptual understanding are the 
same and should be the goal of instruction. 

The National Council of Teachers of Mathematics (NCTM) in it's 

1989 publication Curriculum and Evaluation Stan dards for — School 

Mathematics (commonly referred to as the Standards ) fosters conceptual 
mathematical learning and relates it to multiple representations of 
the graphing calculator: 

The 9-12 standards call for a shift in emphasis from a curriculum 
dominated by memorization of isolated facts and procedures and by 
proficiency with paper-and-pencil skills to one that emphasizes 
conceptual understanding, multiple representations and 
connections, mathematical modeling, and mathematical problem 
solving. The integration of ideas from algebra and geometry is 
particularly strong, with graphical representation playing a 
connecting role. Thus, frequent reference to graphing utilities 
will be found throughout these standards. (1989, p. 125). 
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The graphing calculator which is often referred to as a hand— held 
computer shows promise in recent research as a tool to assist the 
learner construct conceptual knowledge in mathematics in the areas of 
algebra and functions (Estes, 1990; Shoaf-Grubbs , 1992; Tolias, 1993). 
Graphing calculators have an advantage over computers in mathematics 
classrooms in their lower cost and smaller size for portability. 

Because of these advantages, the graphing calculator has gained 
widespread acceptance as a powerful tool for mathematics classrooms 
(Dick, 1992; Wilson & Krapfl, 1994). Although the use of graphing 
calculators has become extensive in high school, community college, 
and university mathematics classrooms in , the last few years, little is 
known about how and why graphing calculators make a difference in 
mathematical understanding. Much of the initial research has been in 
the form of comparisons of achievement and/or attitudes for groups 
using calculators and traditional, non-calculator groups (Wilson & 
Krapfl, 1994). These studies of educational evaluation of the impact 
of the graphing calculator on the teaching and learning of mathematics 
are not educational research which add knowledge about how or why 
graphing calculators facilitate student construction of mathematical 
knowledge (Bright & Williams, 1994) . "For research effectively to 
guide curriculum development and instruction, we need to find out why" 
(Dunham & Dick, 1994, p. 440). 

Conceptual Framework and Hypotheses 

Several factors that have been associated with performance in 
traditional mathematics have also been mentioned by researchers as 
possible links to achievement by students using graphing calculators. 
The relations of gender, spatial visualization, and mathematical 
confidence to traditional mathematical performance have been heavily 
investigated. Such factors show promise for exploratory correlational 
research in graphing calculator utilization. Some studies indicate 
that women are not disadvantaged by the integration of graphing 
calculator technology and in some instances outperform males (Boers 
and Jones, 1992; Dunham, 1990; Ruthven, 1990). Since this achievement 
by females is contrary to previous gender studies in non-technology 
mathematics, gender is a variable that needs further investigation. 
Because of the visual representations produced by the graphing 
calculator, spatial visualization has been suggested as a factor for 
research. Whether the calculator increases spatial ability as Shoaf- 
Grubs (1993) reported or offers "an alternative source of visual 
images for those who cannot create their own" (Dunham, 1995) , spatial 
visualization as a factor related to mathematical performance with 
graphing calculator utilization merits further study. While Dunham 
(1990) reports gains in confidence for students in graphing 
calculator classes, Shoaf-Grubbs (1993) describes comments of 
increased confidence from female students with the graphing calculator 
as a tool for checking solutions. Is this confidence with using 
technology related to performance with the technology? Thus gender, 
spatial visualization, and confidence are prominent factors for 
further study in relation to conceptual mathematical performance in 
calculator-enhanced algebra instruction. 

Other factors that may be related to performance in graphing 
calculator-enhanced classrooms are basic algebra ability and classroom 
graphing calculator utilization. Understandably, a student's basic 
algebra ability will influence her performance in studying algebra 
with or without the enhancement of technology. How strong is that 



relationship when graphing calculators are utilized? Because of 
classroom variation in the utilization of the calculator, when 
examining multiple classrooms, this information should be collected as 
a possible factor for further study. 

Research Question 

The research question guiding this study is: 

What is the relation of the factors of gender, spatial 
visualization, mathematical confidence, basic algebra ability, 
and classroom graphing calculator utilization to conceptual 
mathematical performance with graphing calculators in college 

algebra? . . 

Five proposed hypotheses associated with this question are: 

#1. There is a positive relation between Basic Algebra Ability 
and Conceptual Mathematical Performance. 

#2. There is no relation between Gender and Conceptual 
Mathematical Performance. • 

#3. There is a positive relation between Classroom Graphing 
Calculator Utilization and Conceptual Mathematical Performance. 

#4. There is a positive relation between Spatial Visualization 
and Conceptual Mathematical Performance. 

#5. There is a positive relation between Mathematical Confidence 
and Conceptual Mathematical Performance. 

Methods 

Sample 

The study consisted of the quantitative analysis of the five 
factors of gender, spatial visualization, mathematical confidence, 
basic algebra ability, and classroom graphing calculator utilization 
through correlational statistical techniques. 

This study was conducted at a major Rocky Mountain university 
during the Spring semester of 1996. The subjects were undergraduate 
students enrolled in one of the nine sections of Math 1400 — College 
Algebra who completed all four of the measurements administered for 
this study. This was the second year that graphing calculators were 
required for this course. Students must meet prerequisites for 
admission to this course through one of the following: 

1. Grade of C or better in a prerequisite course (a non-credit 
elementary algebra) 

2. Sufficiently high ACT/SAT score in math 

3. Performance on the university Math Placement Exam passing 
arithmetic, elementary algebra, and intermediate algebra 

Students in College Algebra are generally classified as freshmen, 
sophomores, or juniors. The majority of the students need to take 
additional math coursework in calculus. 

Students in all sections used the same textbook, had the same 
assignment of topics and suggested homework, and took the same exam at 
the same time. The textbook used for the course was College Algebra: 

A Graphics Approach by M. G. Settle (1995) . It was selected by the 
previous year's instructors who wanted a textbook that included 
specific calculator activities. Purchase of a graphing calculator 
such as the TI-81 was recommended for all students, and this model was 
depicted in the textbook with instructions for operation. Instructors 
also used this model for demonstrations in class with an overhead 
projection device and large screen. 

Instruments . . 

For correlational analysis, the following tests were administered 
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1. Spatial visualization — Paper Folding Test, VZ-2 of the Ki t: 
of Factor-Referenced Cognitive Tests published by 
Educational Testing Services. 

2. The Mathematical Self-Concept Scale by Annette Gourgey. 

Student performance on the first class exam, which was a review 

of basic algebra covered in the first three weeks of class with only 
very minimal calculator utilization, served as the basic mathematical 

ability variable measure. ... . 

To guantify the factor of classroom utilization of the graphing 
calculator, the researcher and two assistants observed classrooms 
three times during the period of intensive study of functions 
utilizing graphing calculators. Observers rated the technology 
utilization in the classrooms according to researcher-developed 

guidelines (see Appendix A) . . 

Students' scores on the third class exam, after nine weeks of 
intensive study of functions, were divided into subscales for the 
mathematical cognitive levels of procedural and conceptual. The 
subscale for conceptual knowledge was the Conceptual Mathematical 
Performance measure. The correlation coefficients for the relations 
of gender, spatial visualization, mathematical confidence, basic 
algebra ability, and classroom graphing calculator utilization to 
Conceptual Mathematical Performance were determined. 

Results 

Descriptive Statistics 

Complete data on all of the measures was collected for 144 
students in College Algebra with 65 male (45%) and 78 female (54%) 

participants. . . 

The cumulative ratings by the three observers appear in Figure 1 
to depict the Classroom Graphing Calculator Utilization measure for 
the nine sections of Math 1400. Each section was observed on the same 
three class meeting days with one observation per rater whenever 
possible. Individual observations were scored on a scale from one to 
five using the guidelines in Appendix A. Rating guidelines included 
both instructional and student utilization of the technology. The 
lowest possible cumulative rating of 3 would occur when the three ^ 
individual ratings were each at the Negative Utilization level, with a 
score of 6 representing consistent Minimal Utilization ratings, 9 for 
consistent Moderate Utilization ratings, 12 for consistent Frequent 
Utilization ratings, and the highest cumulative rating of 15 for 
consistent Intensive Utilization ratings. All nine sections received 
cumulative ratings that were between 6 and 9, inclusive, indicating 
little variation in utilization of the graphing calculators in the 
Minimal to Moderate range across the nine sections. 

Table 1 presents the mean, standard deviation, minimum, and 
maximum scores for the measurements of the variables. Basic Algebra 
Ability (BSCALG) scores are the results of the first exam in College 
Algebra administered early in the course after three weeks of review 
before the introduction of functions and the utilization of the 
graphing calculator for graphing purposes. The measure for Classroom 
Graphing Calculator Utilization (CLSUTL) was the ratings reported for 
each section in Figure 1. 
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Cummulative Observation by Section 
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Table 1 



Descriptive Statistics for Variables. — (n=144 1 

' Scored Possible 

Variable Mean Std Dev Min Max Min Max 



BSCALG 
CLSUTL 
SPAVIS 
MCONF 
PRCDMPRF 
CNCMPRF 

NOTE: The labels with the factor represented in parentheses are as 

follows: BSCALG (Basic Algebra Ability) , CLSUTL (Classroom Graphing 

Calculator Utilization) , SPAVIS (Spatial Visualization) , MCONF 
(Mathematical Confidence), PRCDMPRF (Procedural Mathematical 
Performance) , and CNCMPRF (Conceptual Mathematical Performance) . 
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The instrument used to measure Spatial Visualization (SPAVIS) .was the 
Paper Folding Test: VZ-2 of the Kit of Factor-Referenced Cogni tive 
Tests (1976 [Reprinted 1995]). Mathematical Confidence was measured 
using Gourgey's Mathematical Self - Concept Scale (1982) . Procedural 
Mathematical Performance and Conceptual Mathematical Performance are 
the two subscales of the third exam in College Algebra after eight 
weeks of study of functions enhanced with the utilization of the 
graphing calculator. These subscales were obtained by dividing the 
questions of the third exam as to whether they were procedural 
knowledge items or conceptual knowledge items as determined by an 

expert panel. .... 

The results reported in Table 1 for Spatial Visualization and 
Mathematical Confidence for the sample population of College Algebra 
students tested in this study is consistent with data from previous 
studies. The mean of 12.82 for the sample population for spatial 
visualization using the Educational Testing Service's Paper Folding 
Test: VZ-2 is within the range of mean scores reported for the norming 
populations in the Manual for Kit of Fa ctor-Referenced — Cognitive Test s 
(1976). When Gourgey developed the Scale for the Measurement of Self- 
Concept in Mathematics, her sample population scored a mean of 94.53 
with a standard deviation of 21.88 (range of 34 to 133) which are 
similar to the measurements for the college algebra students in this 
study where the mean was 87.94 with a standard deviation of 20.87. 
Correlational Analysis 

Using the statistical package of SPSS, the Pearson's Product- 
Moment Correlation Coefficient, r, was calculated for each of the 
independent variables in relation to the dependent variable Conceptual 
Mathematical Performance. In addition, the intercorrelations between 





the independent variables and with the Procedural Mathematical 
Performance measure were computed and appear in Table 2 . The 
independent measures of Basic Algebra Ability, Spatial Visualization, 
and Mathematical Confidence exhibit significant statistical 
correlation to the dependent measure of Conceptual Mathematical 
Performance. Two other measures of mathematics abilities, the Basic 
Algebra Ability measure and the Procedural Mathematical Performance, 
are also correlated to Spatial Visualization and Mathematical 
Confidence. 

Research Questions 

The results of the correlational analysis were used to determine 
whether to retain or reject the five research hypotheses stated as 
null hypotheses as follows: 

#1. There is no relation between Basic Algebra Ability and 
Conceptual Mathematical Performance. Rejected (p < .01). The 
original hypothesis that there would be a positive relation holds. 

Question #2. There is no relation between Gender and Conceptual 
Mathematical Performance. Retained (p > .05). The original 
hypothesis that there would be no correlation holds. 

Question #3. There is no relation between Classroom Graphing 
Calculator Utilization and Conceptual Mathematical Performance. 
Retained (p > .05). The original hypothesis thatthey would be 
positively correlated did not hold. 

Question #4. There is no relation between Spatial Visualization 
and Conceptual Mathematical Performance. Rejected 
they would be positively correlated did not hold. (p < .05). The 
original hypothesis that they would be positively correlated holds. 

Question #5. There is no relation between Mathematical 
Confidence and Conceptual Mathematical Performance. Rejected (p < 

.01) . The original hypothesis that they would be positively 
correlated holds. 

Discussion and Educational Implications 

The question guiding this study was: What is the relation of the 

factors of gender, spatial visualization, mathematical confidence, 
basic algebra ability, and classroom graphing calculator utilization 
to conceptual mathematical performance with graphing calculators in 
college algebra? 

Looking at the lack of correlation of classroom graphing 
calculator utilization one possible explanations stands out. The 
results of the ratings of the observations for the classroom graphing 
calculator utilization showed little variation over the rating scale 
with scores only ranging from 6 to 9 inclusive on a scale of possible 
values from 3 to 15. This closeness in ratings leaves little 
opportunity to test the correlation of this variable to the dependent 
variable, conceptual mathematical performance. 

The levels of the ratings for classroom graphing calculator 
utilization also explain an important aspect of this study in relation 
to a reform process in the teaching of college algebra that was just 
getting under way at this particular institution. All ratings are in 
the Minimum range up to the baseline of the Moderate range on the 
scale. This indicates that the integration of this technology was not 
yet a major part of the instructional process. The data collected in 
this study may serve as a baseline for data collected in the future as 
the reform process changes the teaching of college algebra. 

The variable Gender did not correlate with Conceptual 





Correlations of the individual variables. 
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Mathematical Performance, Basic Algebra Ability, Spatial 
Visualization, and Mathematical Confidence. Because students must 
meet rigorous prerequisites to enter the class, they enter college 
algebra with similar abilities which may have eliminated some of the 
gender differences seen in early mathematical ability studies. This 
lack of gender correlations supports the equity of graphing 
calculator-enhanced instructional practices in college algebra. 

The results reveal that the variables of spatial visualization 
and mathematical confidence are related to conceptual mathematical 
performance when graphing calculators are utilized as a tool. This 
information is important in directing further investigation for the 
purpose of curriculum and instructional development. 
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Appendix A 



Guidelines and Procedures 
for Classroom Calculator Utilization Observations 

The purpose of observations of the classroom is to determine the 
level of utilization of the graphing calculators by the instructor and 
students. The rating .system in no way tries to judge the teaching of 
the instructor, only the guantity and quality of graphing calculator 
utilization occurring by both the students and instructor in a 
classroom at a particular moment in time. The highest and lowest 
ratings are extremes that will probably not exist in the sections of 
college algebra but are included to realistically identify all 
possible levels that exist. A panel of experts provided comments on 
initial drafts of the rating scale. Three raters simultaneously 
scored two sessions and then compared scoring to develop additional 
guidelines to insure reliability in ratings. 



Each section will be observed and categorized according to the 
five-level rating system of graphing calculator utilization on three 
different class meetings. The dates for classroom observations were 
specifically chosen to coincide with instruction on topics concerning 
functions that would present opportunities for possible intensive use 
of the graphing calculators. 



A point system will used so that individual ratings can then be 
cumulated for comparison. The scale for points is: 

Level Points 

Intensive 5 

Frequent 4 

Moderate 3 

Minimum 2 

Negative 1 



In an Intensive graphing calculator utilization classroom, the 
instructor would thoroughly integrate the use of the calculator to 
enhance learning as a natural part of the curriculum. When discussing 
problem solving and new concepts, the instructor would emphasize 
multiple solution methods which would include both algebraic and 
calculator solutions and the recall of graphical images as an integral 
part of instruction. The instructor might demonstrate new techniques 
with the calculator and encourage students to use their own 
calculators to follow along. The instructor would encourage students 
to help each other in utilizing the calculators. Throughout the 
class, students frequently would be engaged in working with their own 
calculators. Calculators would be utilized in numerous way including 
numerical calculations, algebraic substitutions, graphing, and 
exploration for conceptual development. 

In a Frequent graphing calculator utilization classroom, the 
instructor would find frequent opportunities to utilize the graphing 
calculator in several ways for concept development and problem 
solving. Students would utilize their calculators on numerous 



occasions and be encouraged to do so by the instructor. 



In a Moderate graphing calculator utilization classroom, the 
instructor would utilize the calculator for some concept development 
and problem solving while only using algebraic or other methods at 
other times even though the calculator could be utilized (thus missing 
opportunities for multiple methods) . The instructor might work with 
calculator problem solutions mostly as a tool for checking or simple 
solutions, but not for multiple types of uses nor for exploration. 

The students would use their calculators moderately, simply listening 
or watching displays rather than working along on some occasions. 

In a Minimal graphing calculator utilization) classroom the 
instructor would utilize the graphing calculator minimally by missing 
numerous opportunities for multiple methods of solutions that included 
calculators (as though the instructor just didn't understand how to 
utilize this tool for mathematical learning) . The students would 
seldom use their calculators except for simple calculations. 

In a Negative graphing calculator utilization classroom, the 
instructor would discourage calculator use in instruction. The 
instructor might refuse to help students who ask questions about how 
to manipulate the calculator and become upset with students who try to 
help each other with calculator utilization in class. When discussing 
problem solving, the instructor would only discuss non-calculator 
solutions whenever possible and convey a negative attitude when 
needing to use the calculator. Students would seldom bother to use 
their own calculators in class. 
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